Apr 14 04:12:15 user nova-compute[71317]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code.
Apr 14 04:12:17 user nova-compute[71317]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_linux_bridge.linux_bridge.LinuxBridgePlugin'>' with name 'linux_bridge' {{(pid=71317) initialize /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:44}}
Apr 14 04:12:17 user nova-compute[71317]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' {{(pid=71317) initialize /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:44}}
Apr 14 04:12:17 user nova-compute[71317]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_ovs.ovs.OvsPlugin'>' with name 'ovs' {{(pid=71317) initialize /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:44}}
Apr 14 04:12:17 user nova-compute[71317]: INFO os_vif [-] Loaded VIF plugins: linux_bridge, noop, ovs
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): grep -F node.session.scan /sbin/iscsiadm {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "grep -F node.session.scan /sbin/iscsiadm" returned: 0 in 0.020s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:12:18 user nova-compute[71317]: INFO nova.virt.driver [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] Loading compute driver 'libvirt.LibvirtDriver'
Apr 14 04:12:18 user nova-compute[71317]: INFO nova.compute.provider_config [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] No provider configs found in /etc/nova/provider_config/. If files are present, ensure the Nova process has access.
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] Acquiring lock "singleton_lock" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] Acquired lock "singleton_lock" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] Releasing lock "singleton_lock" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] Full set of CONF: {{(pid=71317) _wait_for_exit_or_signal /usr/local/lib/python3.10/dist-packages/oslo_service/service.py:362}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ******************************************************************************** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2589}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] Configuration options gathered from: {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2590}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] command line args: ['--config-file', '/etc/nova/nova-cpu.conf'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2591}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] config files: ['/etc/nova/nova-cpu.conf'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2592}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ================================================================================ {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2594}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] allow_resize_to_same_host      = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] arq_binding_timeout            = 300 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] backdoor_port                  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] backdoor_socket                = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] block_device_allocate_retries  = 300 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] block_device_allocate_retries_interval = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cert                           = self.pem {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute_driver                 = libvirt.LibvirtDriver {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute_monitors               = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] config_dir                     = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] config_drive_format            = iso9660 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] config_file                    = ['/etc/nova/nova-cpu.conf'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] config_source                  = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] console_host                   = user {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] control_exchange               = nova {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cpu_allocation_ratio           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] daemon                         = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] debug                          = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] default_access_ip_network_name = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] default_availability_zone      = nova {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] default_ephemeral_format       = ext4 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] default_schedule_zone          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] disk_allocation_ratio          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] enable_new_services            = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] enabled_apis                   = ['osapi_compute'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] enabled_ssl_apis               = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] flat_injected                  = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] force_config_drive             = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] force_raw_images               = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] graceful_shutdown_timeout      = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] heal_instance_info_cache_interval = 60 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] host                           = user {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] initial_cpu_allocation_ratio   = 4.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] initial_disk_allocation_ratio  = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] initial_ram_allocation_ratio   = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] injected_network_template      = /opt/stack/nova/nova/virt/interfaces.template {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] instance_build_timeout         = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] instance_delete_interval       = 300 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] instance_format                = [instance: %(uuid)s]  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] instance_name_template         = instance-%08x {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] instance_usage_audit           = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] instance_usage_audit_period    = month {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] instances_path                 = /opt/stack/data/nova/instances {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] internal_service_availability_zone = internal {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] key                            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] live_migration_retry_count     = 30 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] log_config_append              = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] log_dir                        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] log_file                       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] log_options                    = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] log_rotate_interval            = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] log_rotate_interval_type       = days {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] log_rotation_type              = none {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] long_rpc_timeout               = 1800 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] max_concurrent_builds          = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] max_concurrent_live_migrations = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] max_concurrent_snapshots       = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] max_local_block_devices        = 3 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] max_logfile_count              = 30 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] max_logfile_size_mb            = 200 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] maximum_instance_delete_attempts = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] metadata_listen                = 0.0.0.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] metadata_listen_port           = 8775 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] metadata_workers               = 3 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] migrate_max_retries            = -1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] mkisofs_cmd                    = genisoimage {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] my_block_storage_ip            = 10.0.0.210 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] my_ip                          = 10.0.0.210 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] network_allocate_retries       = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] osapi_compute_listen           = 0.0.0.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] osapi_compute_listen_port      = 8774 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] osapi_compute_unique_server_name_scope =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] osapi_compute_workers          = 3 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] password_length                = 12 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] periodic_enable                = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] periodic_fuzzy_delay           = 60 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] pointer_model                  = ps2mouse {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] preallocate_images             = none {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] publish_errors                 = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] pybasedir                      = /opt/stack/nova {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ram_allocation_ratio           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] rate_limit_burst               = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] rate_limit_except_level        = CRITICAL {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] rate_limit_interval            = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] reboot_timeout                 = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] reclaim_instance_interval      = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] record                         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] reimage_timeout_per_gb         = 20 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] report_interval                = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] rescue_timeout                 = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] reserved_host_cpus             = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] reserved_host_disk_mb          = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] reserved_host_memory_mb        = 512 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] reserved_huge_pages            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] resize_confirm_window          = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] resize_fs_using_block_device   = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] resume_guests_state_on_host_boot = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] rootwrap_config                = /etc/nova/rootwrap.conf {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] rpc_response_timeout           = 60 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] run_external_periodic_tasks    = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] running_deleted_instance_action = reap {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] running_deleted_instance_poll_interval = 1800 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] running_deleted_instance_timeout = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] scheduler_instance_sync_interval = 120 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] service_down_time              = 60 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] servicegroup_driver            = db {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] shelved_offload_time           = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] shelved_poll_interval          = 3600 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] shutdown_timeout               = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] source_is_ipv6                 = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ssl_only                       = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] state_path                     = /opt/stack/data/nova {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] sync_power_state_interval      = 600 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] sync_power_state_pool_size     = 1000 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] syslog_log_facility            = LOG_USER {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] tempdir                        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] timeout_nbd                    = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] transport_url                  = **** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] update_resources_interval      = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] use_cow_images                 = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] use_eventlog                   = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] use_journal                    = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] use_json                       = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] use_rootwrap_daemon            = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] use_stderr                     = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] use_syslog                     = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vcpu_pin_set                   = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plugging_is_fatal          = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plugging_timeout           = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] virt_mkfs                      = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] volume_usage_poll_interval     = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] watch_log_file                 = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] web                            = /usr/share/spice-html5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_concurrency.disable_process_locking = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_concurrency.lock_path     = /opt/stack/data/nova {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.auth_strategy              = keystone {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.compute_link_prefix        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.dhcp_domain                = novalocal {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.enable_instance_password   = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.glance_link_prefix         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.instance_list_cells_batch_fixed_size = 100 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.instance_list_cells_batch_strategy = distributed {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.instance_list_per_project_cells = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.list_records_by_skipping_down_cells = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.local_metadata_per_cell    = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.max_limit                  = 1000 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.metadata_cache_expiration  = 15 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.neutron_default_tenant_id  = default {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.use_forwarded_for          = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.use_neutron_default_nets   = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.vendordata_dynamic_connect_timeout = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.vendordata_dynamic_failure_fatal = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.vendordata_dynamic_read_timeout = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.vendordata_dynamic_ssl_certfile =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.vendordata_dynamic_targets = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.vendordata_jsonfile_path   = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api.vendordata_providers       = ['StaticJSON'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.backend                  = dogpile.cache.memcached {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.backend_argument         = **** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.config_prefix            = cache.oslo {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.dead_timeout             = 60.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.debug_cache_backend      = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.enable_retry_client      = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.enable_socket_keepalive  = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.enabled                  = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.expiration_time          = 600 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.hashclient_retry_attempts = 2 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.hashclient_retry_delay   = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.memcache_dead_retry      = 300 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.memcache_password        =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.memcache_pool_maxsize    = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.memcache_sasl_enabled    = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.memcache_servers         = ['localhost:11211'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.memcache_socket_timeout  = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.memcache_username        =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.proxies                  = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.retry_attempts           = 2 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.retry_delay              = 0.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.socket_keepalive_count   = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.socket_keepalive_idle    = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.socket_keepalive_interval = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.tls_allowed_ciphers      = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.tls_cafile               = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.tls_certfile             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.tls_enabled              = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cache.tls_keyfile              = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.auth_section            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.auth_type               = password {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.cafile                  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.catalog_info            = volumev3::publicURL {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.certfile                = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.collect_timing          = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.cross_az_attach         = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.debug                   = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.endpoint_template       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.http_retries            = 3 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.insecure                = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.keyfile                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.os_region_name          = RegionOne {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.split_loggers           = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cinder.timeout                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute.consecutive_build_service_disable_threshold = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute.cpu_dedicated_set      = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute.cpu_shared_set         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute.image_type_exclude_list = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute.live_migration_wait_for_vif_plug = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute.max_concurrent_disk_ops = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute.max_disk_devices_to_attach = -1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute.resource_provider_association_refresh = 300 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute.shutdown_retry_interval = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] compute.vmdk_allowed_types     = ['streamOptimized', 'monolithicSparse'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] conductor.workers              = 3 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] console.allowed_origins        = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] console.ssl_ciphers            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] console.ssl_minimum_version    = default {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] consoleauth.token_ttl          = 600 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.cafile                  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.certfile                = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.collect_timing          = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.connect_retries         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.connect_retry_delay     = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.endpoint_override       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.insecure                = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.keyfile                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.max_version             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.min_version             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.region_name             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.service_name            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.service_type            = accelerator {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.split_loggers           = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.status_code_retries     = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.status_code_retry_delay = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.timeout                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.valid_interfaces        = ['internal', 'public'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] cyborg.version                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.backend               = sqlalchemy {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.connection            = **** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.connection_debug      = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.connection_parameters =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.connection_recycle_time = 3600 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.connection_trace      = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.db_inc_retry_interval = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.db_max_retries        = 20 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.db_max_retry_interval = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.db_retry_interval     = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.max_overflow          = 50 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.max_pool_size         = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.max_retries           = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.mysql_enable_ndb      = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.mysql_wsrep_sync_wait = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.pool_timeout          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.retry_interval        = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.slave_connection      = **** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] database.sqlite_synchronous    = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.backend           = sqlalchemy {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.connection        = **** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.connection_debug  = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.connection_parameters =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.connection_recycle_time = 3600 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.connection_trace  = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.db_inc_retry_interval = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.db_max_retries    = 20 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.db_max_retry_interval = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.db_retry_interval = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.max_overflow      = 50 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.max_pool_size     = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.max_retries       = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.mysql_enable_ndb  = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.mysql_sql_mode    = TRADITIONAL {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.mysql_wsrep_sync_wait = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.pool_timeout      = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.retry_interval    = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.slave_connection  = **** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] api_database.sqlite_synchronous = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] devices.enabled_mdev_types     = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ephemeral_storage_encryption.enabled = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ephemeral_storage_encryption.key_size = 512 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.api_servers             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.cafile                  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.certfile                = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.collect_timing          = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.connect_retries         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.connect_retry_delay     = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.debug                   = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.default_trusted_certificate_ids = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.enable_certificate_validation = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.enable_rbd_download     = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.endpoint_override       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.insecure                = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.keyfile                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.max_version             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.min_version             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.num_retries             = 3 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.rbd_ceph_conf           =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.rbd_connect_timeout     = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.rbd_pool                =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.rbd_user                =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.region_name             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.service_name            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.service_type            = image {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.split_loggers           = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.status_code_retries     = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.status_code_retry_delay = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.timeout                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.valid_interfaces        = ['internal', 'public'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.verify_glance_signatures = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] glance.version                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] guestfs.debug                  = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.config_drive_cdrom      = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.config_drive_inject_password = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.dynamic_memory_ratio    = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.enable_instance_metrics_collection = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.enable_remotefx         = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.instances_path_share    =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.iscsi_initiator_list    = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.limit_cpu_features      = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.mounted_disk_query_retry_count = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.power_state_check_timeframe = 60 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.power_state_event_polling_interval = 2 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.qemu_img_cmd            = qemu-img.exe {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.use_multipath_io        = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.volume_attach_retry_count = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.volume_attach_retry_interval = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.vswitch_name            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] hyperv.wait_soft_reboot_seconds = 60 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] mks.enabled                    = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] mks.mksproxy_base_url          = http://127.0.0.1:6090/ {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] image_cache.manager_interval   = 2400 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] image_cache.precache_concurrency = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] image_cache.remove_unused_base_images = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] image_cache.subdirectory_name  = _base {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.api_max_retries         = 60 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.api_retry_interval      = 2 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.auth_section            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.auth_type               = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.cafile                  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.certfile                = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.collect_timing          = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.connect_retries         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.connect_retry_delay     = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.endpoint_override       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.insecure                = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.keyfile                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.max_version             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.min_version             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.partition_key           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.peer_list               = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.region_name             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.serial_console_state_timeout = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.service_name            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.service_type            = baremetal {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.split_loggers           = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.status_code_retries     = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.status_code_retry_delay = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.timeout                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.valid_interfaces        = ['internal', 'public'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ironic.version                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] key_manager.backend            = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] key_manager.fixed_key          = **** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.auth_endpoint         = http://localhost/identity/v3 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.barbican_api_version  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.barbican_endpoint     = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.barbican_endpoint_type = public {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.barbican_region_name  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.cafile                = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.certfile              = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.collect_timing        = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.insecure              = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.keyfile               = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.number_of_retries     = 60 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.retry_delay           = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.send_service_user_token = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.split_loggers         = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.timeout               = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.verify_ssl            = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican.verify_ssl_path       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican_service_user.auth_section = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican_service_user.auth_type = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican_service_user.cafile   = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican_service_user.certfile = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican_service_user.collect_timing = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican_service_user.insecure = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican_service_user.keyfile  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican_service_user.split_loggers = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] barbican_service_user.timeout  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.approle_role_id          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.approle_secret_id        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.cafile                   = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.certfile                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.collect_timing           = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.insecure                 = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.keyfile                  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.kv_mountpoint            = secret {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.kv_version               = 2 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.namespace                = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.root_token_id            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.split_loggers            = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.ssl_ca_crt_file          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.timeout                  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.use_ssl                  = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vault.vault_url                = http://127.0.0.1:8200 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.cafile                = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.certfile              = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.collect_timing        = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.connect_retries       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.connect_retry_delay   = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.endpoint_override     = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.insecure              = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.keyfile               = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.max_version           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.min_version           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.region_name           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.service_name          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.service_type          = identity {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.split_loggers         = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.status_code_retries   = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.status_code_retry_delay = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.timeout               = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.valid_interfaces      = ['internal', 'public'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] keystone.version               = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.connection_uri         =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.cpu_mode               = custom {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.cpu_model_extra_flags  = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: WARNING oslo_config.cfg [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] Deprecated: Option "cpu_model" from group "libvirt" is deprecated. Use option "cpu_models" from group "libvirt".
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.cpu_models             = ['Nehalem'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.cpu_power_governor_high = performance {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.cpu_power_governor_low = powersave {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.cpu_power_management   = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.cpu_power_management_strategy = cpu_state {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.device_detach_attempts = 8 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.device_detach_timeout  = 20 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.disk_cachemodes        = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.disk_prefix            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.enabled_perf_events    = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.file_backed_memory     = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.gid_maps               = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.hw_disk_discard        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.hw_machine_type        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.images_rbd_ceph_conf   =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.images_rbd_glance_store_name =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.images_rbd_pool        = rbd {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.images_type            = default {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.images_volume_group    = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.inject_key             = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.inject_partition       = -2 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.inject_password        = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.iscsi_iface            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.iser_use_multipath     = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_bandwidth = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_completion_timeout = 800 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_downtime = 500 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_downtime_delay = 75 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_downtime_steps = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_inbound_addr = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_permit_auto_converge = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_permit_post_copy = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_scheme  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_timeout_action = abort {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_tunnelled = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: WARNING oslo_config.cfg [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] Deprecated: Option "live_migration_uri" from group "libvirt" is deprecated for removal (
Apr 14 04:12:18 user nova-compute[71317]: live_migration_uri is deprecated for removal in favor of two other options that
Apr 14 04:12:18 user nova-compute[71317]: allow to change live migration scheme and target URI: ``live_migration_scheme``
Apr 14 04:12:18 user nova-compute[71317]: and ``live_migration_inbound_addr`` respectively.
Apr 14 04:12:18 user nova-compute[71317]: ).  Its value may be silently ignored in the future.
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_uri     = qemu+ssh://stack@%s/system {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.live_migration_with_native_tls = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.max_queues             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.mem_stats_period_seconds = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.nfs_mount_options      = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.nfs_mount_point_base   = /opt/stack/data/nova/mnt {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.num_aoe_discover_tries = 3 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.num_iser_scan_tries    = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.num_memory_encrypted_guests = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.num_nvme_discover_tries = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.num_pcie_ports         = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.num_volume_scan_tries  = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.pmem_namespaces        = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.quobyte_client_cfg     = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.rbd_connect_timeout    = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.rbd_destroy_volume_retries = 12 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.rbd_secret_uuid        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.rbd_user               = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.realtime_scheduler_priority = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.remote_filesystem_transport = ssh {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.rescue_image_id        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.rescue_kernel_id       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.rescue_ramdisk_id      = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.rng_dev_path           = /dev/urandom {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.rx_queue_size          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.smbfs_mount_options    =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.snapshot_compression   = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.snapshot_image_format  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.snapshots_directory    = /opt/stack/data/nova/instances/snapshots {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.sparse_logical_volumes = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.swtpm_enabled          = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.swtpm_group            = tss {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.swtpm_user             = tss {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.sysinfo_serial         = unique {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.tx_queue_size          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.uid_maps               = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.use_virtio_for_bridges = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.virt_type              = kvm {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.volume_clear           = zero {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.volume_clear_size      = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.volume_use_multipath   = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.vzstorage_cache_path   = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.vzstorage_log_path     = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.vzstorage_mount_group  = qemu {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.vzstorage_mount_opts   = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.vzstorage_mount_perms  = 0770 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.vzstorage_mount_user   = stack {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] libvirt.wait_soft_reboot_seconds = 120 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.auth_section           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.auth_type              = password {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.cafile                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.certfile               = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.collect_timing         = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.connect_retries        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.connect_retry_delay    = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.default_floating_pool  = public {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.endpoint_override      = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.extension_sync_interval = 600 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.http_retries           = 3 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.insecure               = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.keyfile                = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.max_version            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.metadata_proxy_shared_secret = **** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.min_version            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.ovs_bridge             = br-int {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.physnets               = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.region_name            = RegionOne {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.service_metadata_proxy = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.service_name           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.service_type           = network {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.split_loggers          = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.status_code_retries    = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.status_code_retry_delay = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.timeout                = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.valid_interfaces       = ['internal', 'public'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] neutron.version                = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] notifications.bdms_in_notifications = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] notifications.default_level    = INFO {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] notifications.notification_format = unversioned {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] notifications.notify_on_state_change = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] pci.alias                      = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] pci.device_spec                = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] pci.report_in_placement        = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.auth_section         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.auth_type            = password {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.auth_url             = http://10.0.0.210/identity {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.cafile               = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.certfile             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.collect_timing       = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.connect_retries      = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.connect_retry_delay  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.default_domain_id    = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.default_domain_name  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.domain_id            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.domain_name          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.endpoint_override    = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.insecure             = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.keyfile              = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.max_version          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.min_version          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.password             = **** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.project_domain_id    = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.project_domain_name  = Default {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.project_id           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.project_name         = service {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.region_name          = RegionOne {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.service_name         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.service_type         = placement {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.split_loggers        = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.status_code_retries  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.status_code_retry_delay = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.system_scope         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.timeout              = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.trust_id             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.user_domain_id       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.user_domain_name     = Default {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.user_id              = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.username             = placement {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.valid_interfaces     = ['internal', 'public'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] placement.version              = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.cores                    = 20 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.count_usage_from_placement = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.driver                   = nova.quota.DbQuotaDriver {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.injected_file_content_bytes = 10240 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.injected_file_path_length = 255 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.injected_files           = 5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.instances                = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.key_pairs                = 100 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.metadata_items           = 128 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.ram                      = 51200 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.recheck_quota            = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.server_group_members     = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] quota.server_groups            = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] rdp.enabled                    = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] rdp.html5_proxy_base_url       = http://127.0.0.1:6083/ {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] scheduler.enable_isolated_aggregate_filtering = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] scheduler.image_metadata_prefilter = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] scheduler.max_attempts         = 3 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] scheduler.max_placement_results = 1000 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] scheduler.placement_aggregate_required_for_tenants = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] scheduler.query_placement_for_availability_zone = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] scheduler.query_placement_for_image_type_support = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] scheduler.workers              = 3 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.host_subset_size = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.image_properties_default_architecture = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.isolated_hosts = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.isolated_images = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.max_instances_per_host = 50 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.max_io_ops_per_host = 8 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.pci_in_placement = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.track_instance_changes = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] metrics.required               = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] metrics.weight_multiplier      = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] metrics.weight_of_unavailable  = -10000.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] metrics.weight_setting         = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] serial_console.base_url        = ws://127.0.0.1:6083/ {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] serial_console.enabled         = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] serial_console.port_range      = 10000:20000 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] serial_console.proxyclient_address = 127.0.0.1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] serial_console.serialproxy_host = 0.0.0.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] serial_console.serialproxy_port = 6083 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] service_user.auth_section      = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] service_user.auth_type         = password {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] service_user.cafile            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] service_user.certfile          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] service_user.collect_timing    = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] service_user.insecure          = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] service_user.keyfile           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] service_user.send_service_user_token = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] service_user.split_loggers     = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] service_user.timeout           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] spice.agent_enabled            = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] spice.enabled                  = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] spice.html5proxy_base_url      = http://10.0.0.210:6081/spice_auto.html {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] spice.html5proxy_host          = 0.0.0.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] spice.html5proxy_port          = 6082 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] spice.image_compression        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] spice.jpeg_compression         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] spice.playback_compression     = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] spice.server_listen            = 127.0.0.1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] spice.server_proxyclient_address = 127.0.0.1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] spice.streaming_mode           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] spice.zlib_compression         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] upgrade_levels.baseapi         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] upgrade_levels.cert            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] upgrade_levels.compute         = auto {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] upgrade_levels.conductor       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] upgrade_levels.scheduler       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vendordata_dynamic_auth.auth_section = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vendordata_dynamic_auth.auth_type = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vendordata_dynamic_auth.cafile = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vendordata_dynamic_auth.certfile = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vendordata_dynamic_auth.collect_timing = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vendordata_dynamic_auth.insecure = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vendordata_dynamic_auth.keyfile = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vendordata_dynamic_auth.split_loggers = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vendordata_dynamic_auth.timeout = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.api_retry_count         = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.ca_file                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.cache_prefix            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.cluster_name            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.connection_pool_size    = 10 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.console_delay_seconds   = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.datastore_regex         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.host_ip                 = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.host_password           = **** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.host_port               = 443 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.host_username           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.insecure                = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.integration_bridge      = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.maximum_objects         = 100 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.pbm_default_policy      = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.pbm_enabled             = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.pbm_wsdl_location       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.serial_log_dir          = /opt/vmware/vspc {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.serial_port_proxy_uri   = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.serial_port_service_uri = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.task_poll_interval      = 0.5 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.use_linked_clone        = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.vnc_keymap              = en-us {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.vnc_port                = 5900 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vmware.vnc_port_total          = 10000 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vnc.auth_schemes               = ['none'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vnc.enabled                    = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vnc.novncproxy_base_url        = http://10.0.0.210:6080/vnc_lite.html {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vnc.novncproxy_host            = 0.0.0.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vnc.novncproxy_port            = 6080 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vnc.server_listen              = 0.0.0.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vnc.server_proxyclient_address = 10.0.0.210 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vnc.vencrypt_ca_certs          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vnc.vencrypt_client_cert       = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vnc.vencrypt_client_key        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.disable_compute_service_check_for_ffu = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.disable_fallback_pcpu_query = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.disable_group_policy_check_upcall = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.disable_libvirt_livesnapshot = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.disable_rootwrap   = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.enable_numa_live_migration = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.enable_qemu_monitor_announce_self = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.handle_virt_lifecycle_events = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.libvirt_disable_apic = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.never_download_image_if_on_rbd = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.skip_cpu_compare_at_startup = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.skip_cpu_compare_on_dest = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] wsgi.api_paste_config          = /etc/nova/api-paste.ini {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] wsgi.client_socket_timeout     = 900 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] wsgi.default_pool_size         = 1000 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] wsgi.keep_alive                = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] wsgi.max_header_line           = 16384 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] wsgi.secure_proxy_ssl_header   = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] wsgi.ssl_ca_file               = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] wsgi.ssl_cert_file             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] wsgi.ssl_key_file              = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] wsgi.tcp_keepidle              = 600 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] wsgi.wsgi_log_format           = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] zvm.ca_file                    = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] zvm.cloud_connector_url        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] zvm.image_tmp_path             = /opt/stack/data/nova/images {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] zvm.reachable_timeout          = 300 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_policy.enforce_new_defaults = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_policy.enforce_scope      = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_policy.policy_default_rule = default {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_policy.policy_file        = policy.yaml {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] profiler.connection_string     = messaging:// {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] profiler.enabled               = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] profiler.es_doc_type           = notification {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] profiler.es_scroll_size        = 10000 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] profiler.es_scroll_time        = 2m {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] profiler.filter_error_trace    = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] profiler.hmac_keys             = SECRET_KEY {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] profiler.sentinel_service_name = mymaster {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] profiler.socket_timeout        = 0.1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] profiler.trace_sqlalchemy      = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] remote_debug.host              = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] remote_debug.port              = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.ssl      = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_notifications.retry = -1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_messaging_notifications.transport_url = **** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.auth_section        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.auth_type           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.cafile              = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.certfile            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.collect_timing      = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.connect_retries     = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.connect_retry_delay = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.endpoint_id         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.endpoint_override   = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.insecure            = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.keyfile             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.max_version         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.min_version         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.region_name         = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.service_name        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.service_type        = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.split_loggers       = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.status_code_retries = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.status_code_retry_delay = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.timeout             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.valid_interfaces    = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_limit.version             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_reports.file_event_handler = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] oslo_reports.log_dir           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plug_linux_bridge_privileged.capabilities = [12] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plug_linux_bridge_privileged.group = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plug_linux_bridge_privileged.helper_command = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plug_linux_bridge_privileged.logger_name = oslo_privsep.daemon {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plug_linux_bridge_privileged.thread_pool_size = 12 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plug_linux_bridge_privileged.user = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plug_ovs_privileged.capabilities = [12, 1] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plug_ovs_privileged.group  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plug_ovs_privileged.helper_command = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plug_ovs_privileged.logger_name = oslo_privsep.daemon {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plug_ovs_privileged.thread_pool_size = 12 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] vif_plug_ovs_privileged.user   = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_linux_bridge.flat_interface = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_linux_bridge.forward_bridge_interface = ['all'] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_linux_bridge.iptables_bottom_regex =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_linux_bridge.iptables_drop_action = DROP {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_linux_bridge.iptables_top_regex =  {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_linux_bridge.network_device_mtu = 1500 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_linux_bridge.use_ipv6   = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_linux_bridge.vlan_interface = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_ovs.isolate_vif         = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_ovs.network_device_mtu  = 1500 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_ovs.ovs_vsctl_timeout   = 120 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_ovs.ovsdb_connection    = tcp:127.0.0.1:6640 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_ovs.ovsdb_interface     = native {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_vif_ovs.per_port_bridge     = False {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] os_brick.lock_path             = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] privsep_osbrick.capabilities   = [21] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] privsep_osbrick.group          = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] privsep_osbrick.helper_command = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] privsep_osbrick.logger_name    = os_brick.privileged {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] privsep_osbrick.thread_pool_size = 12 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] privsep_osbrick.user           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] nova_sys_admin.capabilities    = [0, 1, 2, 3, 12, 21] {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] nova_sys_admin.group           = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] nova_sys_admin.helper_command  = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] nova_sys_admin.logger_name     = oslo_privsep.daemon {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] nova_sys_admin.thread_pool_size = 12 {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] nova_sys_admin.user            = None {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG oslo_service.service [None req-e7c7603e-5273-41d3-8e69-c0ca42015ce7 None None] ******************************************************************************** {{(pid=71317) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2613}}
Apr 14 04:12:18 user nova-compute[71317]: INFO nova.service [-] Starting compute node (version 0.0.0)
Apr 14 04:12:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Starting native event thread {{(pid=71317) _init_events /opt/stack/nova/nova/virt/libvirt/host.py:492}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Starting green dispatch thread {{(pid=71317) _init_events /opt/stack/nova/nova/virt/libvirt/host.py:498}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Starting connection event dispatch thread {{(pid=71317) initialize /opt/stack/nova/nova/virt/libvirt/host.py:620}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Connecting to libvirt: qemu:///system {{(pid=71317) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:503}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Registering for lifecycle events <nova.virt.libvirt.host.Host object at 0x7f84cd7b6620> {{(pid=71317) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:509}}
Apr 14 04:12:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Registering for connection events: <nova.virt.libvirt.host.Host object at 0x7f84cd7b6620> {{(pid=71317) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:530}}
Apr 14 04:12:18 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Connection event '1' reason 'None'
Apr 14 04:12:18 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Cannot update service status on host "user" since it is not registered.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host user could not be found.
Apr 14 04:12:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.mount [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Initialising _HostMountState generation 0 {{(pid=71317) host_up /opt/stack/nova/nova/virt/libvirt/volume/mount.py:130}}
Apr 14 04:12:26 user nova-compute[71317]: INFO nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Libvirt host capabilities <capabilities>
Apr 14 04:12:26 user nova-compute[71317]:   <host>
Apr 14 04:12:26 user nova-compute[71317]:     <uuid>e20c3142-5af9-7467-ecd8-70b2e4a210d6</uuid>
Apr 14 04:12:26 user nova-compute[71317]:     <cpu>
Apr 14 04:12:26 user nova-compute[71317]:       <arch>x86_64</arch>
Apr 14 04:12:26 user nova-compute[71317]:       <model>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <vendor>Intel</vendor>
Apr 14 04:12:26 user nova-compute[71317]:       <microcode version='1070'/>
Apr 14 04:12:26 user nova-compute[71317]:       <counter name='tsc' frequency='2094952000' scaling='no'/>
Apr 14 04:12:26 user nova-compute[71317]:       <topology sockets='6' dies='1' cores='1' threads='1'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='ss'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='vmx'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='pcid'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='osxsave'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='hypervisor'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='arat'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='tsc_adjust'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='md-clear'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='arch-capabilities'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='invtsc'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='rsba'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature name='skip-l1dfl-vmentry'/>
Apr 14 04:12:26 user nova-compute[71317]:       <pages unit='KiB' size='4'/>
Apr 14 04:12:26 user nova-compute[71317]:       <pages unit='KiB' size='2048'/>
Apr 14 04:12:26 user nova-compute[71317]:     </cpu>
Apr 14 04:12:26 user nova-compute[71317]:     <power_management>
Apr 14 04:12:26 user nova-compute[71317]:       <suspend_mem/>
Apr 14 04:12:26 user nova-compute[71317]:       <suspend_disk/>
Apr 14 04:12:26 user nova-compute[71317]:       <suspend_hybrid/>
Apr 14 04:12:26 user nova-compute[71317]:     </power_management>
Apr 14 04:12:26 user nova-compute[71317]:     <iommu support='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     <migration_features>
Apr 14 04:12:26 user nova-compute[71317]:       <live/>
Apr 14 04:12:26 user nova-compute[71317]:       <uri_transports>
Apr 14 04:12:26 user nova-compute[71317]:         <uri_transport>tcp</uri_transport>
Apr 14 04:12:26 user nova-compute[71317]:         <uri_transport>rdma</uri_transport>
Apr 14 04:12:26 user nova-compute[71317]:       </uri_transports>
Apr 14 04:12:26 user nova-compute[71317]:     </migration_features>
Apr 14 04:12:26 user nova-compute[71317]:     <topology>
Apr 14 04:12:26 user nova-compute[71317]:       <cells num='2'>
Apr 14 04:12:26 user nova-compute[71317]:         <cell id='0'>
Apr 14 04:12:26 user nova-compute[71317]:           <memory unit='KiB'>8152920</memory>
Apr 14 04:12:26 user nova-compute[71317]:           <pages unit='KiB' size='4'>2038230</pages>
Apr 14 04:12:26 user nova-compute[71317]:           <pages unit='KiB' size='2048'>0</pages>
Apr 14 04:12:26 user nova-compute[71317]:           <distances>
Apr 14 04:12:26 user nova-compute[71317]:             <sibling id='0' value='10'/>
Apr 14 04:12:26 user nova-compute[71317]:             <sibling id='1' value='20'/>
Apr 14 04:12:26 user nova-compute[71317]:           </distances>
Apr 14 04:12:26 user nova-compute[71317]:           <cpus num='6'>
Apr 14 04:12:26 user nova-compute[71317]:             <cpu id='0' socket_id='0' die_id='0' core_id='0' siblings='0'/>
Apr 14 04:12:26 user nova-compute[71317]:             <cpu id='1' socket_id='2' die_id='0' core_id='0' siblings='1'/>
Apr 14 04:12:26 user nova-compute[71317]:             <cpu id='2' socket_id='4' die_id='0' core_id='0' siblings='2'/>
Apr 14 04:12:26 user nova-compute[71317]:             <cpu id='3' socket_id='6' die_id='0' core_id='0' siblings='3'/>
Apr 14 04:12:26 user nova-compute[71317]:             <cpu id='4' socket_id='8' die_id='0' core_id='0' siblings='4'/>
Apr 14 04:12:26 user nova-compute[71317]:             <cpu id='5' socket_id='10' die_id='0' core_id='0' siblings='5'/>
Apr 14 04:12:26 user nova-compute[71317]:           </cpus>
Apr 14 04:12:26 user nova-compute[71317]:         </cell>
Apr 14 04:12:26 user nova-compute[71317]:         <cell id='1'>
Apr 14 04:12:26 user nova-compute[71317]:           <memory unit='KiB'>8255068</memory>
Apr 14 04:12:26 user nova-compute[71317]:           <pages unit='KiB' size='4'>2063767</pages>
Apr 14 04:12:26 user nova-compute[71317]:           <pages unit='KiB' size='2048'>0</pages>
Apr 14 04:12:26 user nova-compute[71317]:           <distances>
Apr 14 04:12:26 user nova-compute[71317]:             <sibling id='0' value='20'/>
Apr 14 04:12:26 user nova-compute[71317]:             <sibling id='1' value='10'/>
Apr 14 04:12:26 user nova-compute[71317]:           </distances>
Apr 14 04:12:26 user nova-compute[71317]:           <cpus num='6'>
Apr 14 04:12:26 user nova-compute[71317]:             <cpu id='6' socket_id='12' die_id='0' core_id='0' siblings='6'/>
Apr 14 04:12:26 user nova-compute[71317]:             <cpu id='7' socket_id='14' die_id='0' core_id='0' siblings='7'/>
Apr 14 04:12:26 user nova-compute[71317]:             <cpu id='8' socket_id='16' die_id='0' core_id='0' siblings='8'/>
Apr 14 04:12:26 user nova-compute[71317]:             <cpu id='9' socket_id='18' die_id='0' core_id='0' siblings='9'/>
Apr 14 04:12:26 user nova-compute[71317]:             <cpu id='10' socket_id='20' die_id='0' core_id='0' siblings='10'/>
Apr 14 04:12:26 user nova-compute[71317]:             <cpu id='11' socket_id='22' die_id='0' core_id='0' siblings='11'/>
Apr 14 04:12:26 user nova-compute[71317]:           </cpus>
Apr 14 04:12:26 user nova-compute[71317]:         </cell>
Apr 14 04:12:26 user nova-compute[71317]:       </cells>
Apr 14 04:12:26 user nova-compute[71317]:     </topology>
Apr 14 04:12:26 user nova-compute[71317]:     <cache>
Apr 14 04:12:26 user nova-compute[71317]:       <bank id='0' level='3' type='both' size='15' unit='MiB' cpus='0'/>
Apr 14 04:12:26 user nova-compute[71317]:       <bank id='2' level='3' type='both' size='15' unit='MiB' cpus='1'/>
Apr 14 04:12:26 user nova-compute[71317]:       <bank id='4' level='3' type='both' size='15' unit='MiB' cpus='2'/>
Apr 14 04:12:26 user nova-compute[71317]:       <bank id='6' level='3' type='both' size='15' unit='MiB' cpus='3'/>
Apr 14 04:12:26 user nova-compute[71317]:       <bank id='8' level='3' type='both' size='15' unit='MiB' cpus='4'/>
Apr 14 04:12:26 user nova-compute[71317]:       <bank id='10' level='3' type='both' size='15' unit='MiB' cpus='5'/>
Apr 14 04:12:26 user nova-compute[71317]:       <bank id='12' level='3' type='both' size='15' unit='MiB' cpus='6'/>
Apr 14 04:12:26 user nova-compute[71317]:       <bank id='14' level='3' type='both' size='15' unit='MiB' cpus='7'/>
Apr 14 04:12:26 user nova-compute[71317]:       <bank id='16' level='3' type='both' size='15' unit='MiB' cpus='8'/>
Apr 14 04:12:26 user nova-compute[71317]:       <bank id='18' level='3' type='both' size='15' unit='MiB' cpus='9'/>
Apr 14 04:12:26 user nova-compute[71317]:       <bank id='20' level='3' type='both' size='15' unit='MiB' cpus='10'/>
Apr 14 04:12:26 user nova-compute[71317]:       <bank id='22' level='3' type='both' size='15' unit='MiB' cpus='11'/>
Apr 14 04:12:26 user nova-compute[71317]:     </cache>
Apr 14 04:12:26 user nova-compute[71317]:     <secmodel>
Apr 14 04:12:26 user nova-compute[71317]:       <model>apparmor</model>
Apr 14 04:12:26 user nova-compute[71317]:       <doi>0</doi>
Apr 14 04:12:26 user nova-compute[71317]:     </secmodel>
Apr 14 04:12:26 user nova-compute[71317]:     <secmodel>
Apr 14 04:12:26 user nova-compute[71317]:       <model>dac</model>
Apr 14 04:12:26 user nova-compute[71317]:       <doi>0</doi>
Apr 14 04:12:26 user nova-compute[71317]:       <baselabel type='kvm'>+64055:+108</baselabel>
Apr 14 04:12:26 user nova-compute[71317]:       <baselabel type='qemu'>+64055:+108</baselabel>
Apr 14 04:12:26 user nova-compute[71317]:     </secmodel>
Apr 14 04:12:26 user nova-compute[71317]:   </host>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='alpha'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>64</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-alpha</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>clipper</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='armv6l'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-arm</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>integratorcp</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>ast2600-evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>borzoi</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>spitz</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.7</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>nuri</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>romulus-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-3.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-5.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>npcm750-evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>rainier-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps3-an547</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>musca-b1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>versatileab</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>kzm</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>musca-a</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-3.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-5.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>smdkc210</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sx1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.11</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>imx25-pdk</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>orangepi-pc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>z2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-5.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>tosa</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an500</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.12</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>mps2-an521</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>sabrelite</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an511</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>canon-a1100</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>realview-eb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>emcraft-sf2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>realview-pb-a8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-4.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>raspi1ap</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>palmetto-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sx1-v1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>n810</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>g220a-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>n800</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>tacoma-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-4.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>quanta-gsj</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>versatilepb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>terrier</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mainstone</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-4.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>mps3-an524</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>kudo-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>vexpress-a9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>midway</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>musicpal</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>lm3s811evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>lm3s6965evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>microbit</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an505</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an385</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-6.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>cubieboard</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>verdex</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>netduino2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an386</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-6.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>raspi2b</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>vexpress-a15</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>fuji-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>cheetah</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.6</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>ast2500-evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>highbank</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>akita</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>connex</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>netduinoplus2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>collie</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>raspi0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='armv7l'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-arm</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>integratorcp</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>ast2600-evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>borzoi</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>spitz</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.7</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>nuri</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>romulus-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-3.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-5.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>npcm750-evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>rainier-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps3-an547</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>musca-b1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>versatileab</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>kzm</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>musca-a</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-3.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-5.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>smdkc210</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sx1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.11</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>imx25-pdk</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>orangepi-pc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>z2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-5.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>tosa</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an500</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.12</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>mps2-an521</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>sabrelite</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an511</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>canon-a1100</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>realview-eb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>emcraft-sf2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>realview-pb-a8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-4.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>raspi1ap</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>palmetto-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sx1-v1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>n810</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>g220a-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>n800</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>tacoma-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-4.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>quanta-gsj</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>versatilepb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>terrier</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mainstone</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-4.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>mps3-an524</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>kudo-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>vexpress-a9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>midway</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>musicpal</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>lm3s811evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>lm3s6965evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>microbit</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an505</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an385</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-6.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>cubieboard</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>verdex</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>netduino2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an386</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-6.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>raspi2b</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>vexpress-a15</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>fuji-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>cheetah</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.6</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>ast2500-evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>highbank</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>akita</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>connex</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>netduinoplus2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>collie</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>raspi0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='aarch64'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>64</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-aarch64</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>integratorcp</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>ast2600-evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>borzoi</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>spitz</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.7</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>nuri</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>romulus-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-3.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-5.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>npcm750-evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>rainier-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps3-an547</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>musca-b1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>versatileab</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>kzm</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>musca-a</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-3.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-5.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>smdkc210</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sx1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.11</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>imx25-pdk</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>orangepi-pc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>z2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-5.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='6'>xlnx-zcu102</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>tosa</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an500</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.12</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>mps2-an521</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>sabrelite</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an511</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>canon-a1100</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>realview-eb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>emcraft-sf2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>realview-pb-a8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>sbsa-ref</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-4.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>raspi1ap</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>palmetto-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sx1-v1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>n810</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>g220a-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>n800</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>tacoma-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-4.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>quanta-gsj</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>versatilepb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>terrier</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mainstone</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-4.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>mps3-an524</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>kudo-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>vexpress-a9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>midway</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>musicpal</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>lm3s811evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>lm3s6965evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>microbit</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an505</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an385</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-6.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>raspi3ap</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>cubieboard</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>verdex</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>netduino2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>xlnx-versal-virt</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mps2-an386</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-6.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>raspi3b</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>raspi2b</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>vexpress-a15</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2'>fuji-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='512'>virt-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>cheetah</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>virt-2.6</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>ast2500-evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>highbank</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>akita</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>connex</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>netduinoplus2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>collie</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>raspi0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <acpi default='on' toggle='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='cris'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-cris</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>axis-dev88</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='i686'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-i386</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-jammy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='pc-i440fx-jammy' maxCpus='255'>ubuntu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-impish-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-5.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.12</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-xenial</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='pc-i440fx-6.2' maxCpus='255'>pc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-4.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.5</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-4.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-focal</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-hirsute</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-q35-xenial</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-jammy-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-5.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-1.5</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-q35-2.7</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-eoan-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-zesty</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-disco-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-groovy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-groovy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-artful</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-trusty</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-eoan-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-focal-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-bionic-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-artful</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.7</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-6.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-yakkety</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-q35-2.4</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-cosmic-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-2.10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>x-remote</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-5.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-1.7</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-2.9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.11</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-3.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-6.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-4.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-jammy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='pc-q35-jammy' maxCpus='288'>ubuntu-q35</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.4</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-4.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-eoan</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-jammy-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-5.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-bionic-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>isapc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-1.4</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-cosmic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-q35-2.6</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-3.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-bionic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-disco-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-cosmic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-2.12</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-bionic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-groovy-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-disco</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-cosmic-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-wily</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-impish</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-6.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-impish</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.6</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-impish-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-hirsute</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-4.0.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-hirsute-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-1.6</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-5.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-2.8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-3.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-6.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-zesty</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-4.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-focal</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>microvm</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.3</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-focal-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-disco</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-4.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-groovy-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-hirsute-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-5.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='pc-q35-6.2' maxCpus='288'>q35</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-eoan</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-q35-2.5</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-3.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-q35-yakkety</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-2.11</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='kvm'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <pae/>
Apr 14 04:12:26 user nova-compute[71317]:       <nonpae/>
Apr 14 04:12:26 user nova-compute[71317]:       <acpi default='on' toggle='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:       <apic default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='m68k'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-m68k</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mcf5208evb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>an5206</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>virt-6.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>q800</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>virt-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='virt-6.2' maxCpus='1'>virt</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>next-cube</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>virt-6.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='microblaze'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-microblaze</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>petalogix-s3adsp1800</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>petalogix-ml605</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>xlnx-zynqmp-pmu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='microblazeel'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-microblazeel</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>petalogix-s3adsp1800</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>petalogix-ml605</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>xlnx-zynqmp-pmu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='mips'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-mips</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='16'>malta</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mipssim</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='mipsel'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-mipsel</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='16'>malta</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mipssim</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='mips64'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>64</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-mips64</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='16'>malta</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mipssim</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>pica61</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>magnum</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='mips64el'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>64</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-mips64el</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='16'>malta</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='16'>loongson3-virt</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mipssim</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>pica61</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>magnum</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='16'>boston</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>fuloong2e</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='ppc'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-ppc</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>g3beige</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>virtex-ml507</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mac99</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>ppce500</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>pegasos2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sam460ex</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>bamboo</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>40p</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>ref405ep</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='15'>mpc8544ds</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>taihu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='ppc64'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>64</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-ppc64</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-jammy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='pseries-jammy' maxCpus='2147483647'>pseries</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2048'>powernv9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='powernv9' maxCpus='2048'>powernv</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>taihu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-4.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='15'>mpc8544ds</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-6.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.5</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2048'>powernv10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-xenial</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-4.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-yakkety</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.6</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>ppce500</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-bionic-sxxm</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.7</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-3.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-5.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>40p</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>pegasos2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-hirsute</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-3.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-5.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-eoan</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-zesty</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>bamboo</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-groovy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-focal</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>g3beige</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-5.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-disco</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.12-sxxm</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>virtex-ml507</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.11</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-cosmic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-bionic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.12</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mac99</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-impish</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-artful</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sam460ex</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>ref405ep</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.3</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2048'>powernv8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-4.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-6.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.4</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='ppc64le'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>64</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-ppc64le</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-jammy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='pseries-jammy' maxCpus='2147483647'>pseries</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2048'>powernv9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='powernv9' maxCpus='2048'>powernv</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>taihu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-4.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='15'>mpc8544ds</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-6.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.5</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2048'>powernv10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-xenial</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-4.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-yakkety</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.6</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>ppce500</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-bionic-sxxm</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.7</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-3.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-5.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>40p</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>pegasos2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-hirsute</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-3.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-5.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-eoan</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-zesty</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>bamboo</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-groovy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-focal</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>g3beige</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-5.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-disco</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.12-sxxm</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>virtex-ml507</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.11</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-cosmic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-bionic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.12</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>mac99</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-impish</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-artful</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sam460ex</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>ref405ep</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.3</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2048'>powernv8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-4.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-6.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='2147483647'>pseries-2.4</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='riscv32'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-riscv32</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='8'>spike</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>opentitan</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='5'>sifive_u</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sifive_e</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='8'>virt</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='riscv64'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>64</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-riscv64</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='8'>spike</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='5'>microchip-icicle-kit</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='5'>sifive_u</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>shakti_c</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sifive_e</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='8'>virt</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='s390x'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>64</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-s390x</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-jammy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='s390-ccw-virtio-jammy' maxCpus='248'>s390-ccw-virtio</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-4.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-5.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-artful</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-3.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-groovy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-hirsute</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-disco</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-2.12</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-2.6</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-yakkety</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-eoan</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-2.9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-6.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-5.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-3.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-4.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-2.5</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-2.11</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-xenial</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-focal</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-2.8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-impish</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-bionic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-5.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-zesty</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-4.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-cosmic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-2.4</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-2.10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-2.7</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='248'>s390-ccw-virtio-6.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='sh4'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-sh4</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>shix</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>r2d</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='sh4eb'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>64</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-sh4eb</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>shix</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>r2d</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='sparc'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-sparc</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>SS-5</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>SS-20</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>LX</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>SPARCClassic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>leon3_generic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>SPARCbook</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>SS-4</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>SS-600MP</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>SS-10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>Voyager</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='sparc64'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>64</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-sparc64</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sun4u</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>niagara</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>sun4v</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='x86_64'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>64</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-jammy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='pc-i440fx-jammy' maxCpus='255'>ubuntu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-impish-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-5.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.12</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-xenial</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='pc-i440fx-6.2' maxCpus='255'>pc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-4.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.5</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-4.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-hirsute</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-focal</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-q35-xenial</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-jammy-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-5.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-1.5</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-q35-2.7</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-eoan-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-zesty</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-disco-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-groovy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-groovy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-artful</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-trusty</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-focal-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-eoan-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-bionic-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-artful</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.7</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-6.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-yakkety</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-q35-2.4</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-cosmic-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-2.10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>x-remote</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-5.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-1.7</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-2.9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.11</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-3.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-6.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-4.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-jammy</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='pc-q35-jammy' maxCpus='288'>ubuntu-q35</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.4</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-4.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-eoan</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-jammy-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-5.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.9</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-bionic-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='1'>isapc</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-1.4</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-cosmic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-q35-2.6</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-3.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-bionic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-disco-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-cosmic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-2.12</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-bionic</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-groovy-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-disco</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-cosmic-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-wily</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-impish</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.6</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-6.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-impish</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-impish-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-hirsute</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-4.0.1</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-hirsute-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-1.6</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-5.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-2.8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.10</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-3.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-zesty</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-4.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-focal</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>microvm</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-6.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.3</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-disco</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-focal-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-4.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-groovy-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-hirsute-hpb</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-5.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-2.8</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine canonical='pc-q35-6.2' maxCpus='288'>q35</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-eoan</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-q35-2.5</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-i440fx-3.0</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='255'>pc-q35-yakkety</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='288'>pc-q35-2.11</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='kvm'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <acpi default='on' toggle='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:       <apic default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='xtensa'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-xtensa</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>sim</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>kc705</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>ml605</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>ml605-nommu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>virt</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>lx60-nommu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>lx200</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>lx200-nommu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>lx60</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>kc705-nommu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]:   <guest>
Apr 14 04:12:26 user nova-compute[71317]:     <os_type>hvm</os_type>
Apr 14 04:12:26 user nova-compute[71317]:     <arch name='xtensaeb'>
Apr 14 04:12:26 user nova-compute[71317]:       <wordsize>32</wordsize>
Apr 14 04:12:26 user nova-compute[71317]:       <emulator>/usr/bin/qemu-system-xtensaeb</emulator>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='4'>sim</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>kc705</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>ml605</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>ml605-nommu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>virt</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>lx60-nommu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>lx200</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>lx200-nommu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>lx60</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <machine maxCpus='32'>kc705-nommu</machine>
Apr 14 04:12:26 user nova-compute[71317]:       <domain type='qemu'/>
Apr 14 04:12:26 user nova-compute[71317]:     </arch>
Apr 14 04:12:26 user nova-compute[71317]:     <features>
Apr 14 04:12:26 user nova-compute[71317]:       <cpuselection/>
Apr 14 04:12:26 user nova-compute[71317]:       <deviceboot/>
Apr 14 04:12:26 user nova-compute[71317]:       <disksnapshot default='on' toggle='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     </features>
Apr 14 04:12:26 user nova-compute[71317]:   </guest>
Apr 14 04:12:26 user nova-compute[71317]: </capabilities>
Apr 14 04:12:26 user nova-compute[71317]: 
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for alpha via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch alpha / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-alpha' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for armv6l via machine types: {'virt', None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch armv6l / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch armv6l / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for armv7l via machine types: {'virt'} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch armv7l / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for aarch64 via machine types: {'virt'} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch aarch64 / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-aarch64' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for cris via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch cris / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-cris' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for i686 via machine types: {'pc', 'q35', 'ubuntu', 'ubuntu-q35'} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=pc:
Apr 14 04:12:26 user nova-compute[71317]: <domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:   <path>/usr/bin/qemu-system-i386</path>
Apr 14 04:12:26 user nova-compute[71317]:   <domain>kvm</domain>
Apr 14 04:12:26 user nova-compute[71317]:   <machine>pc-i440fx-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:   <arch>i686</arch>
Apr 14 04:12:26 user nova-compute[71317]:   <vcpu max='255'/>
Apr 14 04:12:26 user nova-compute[71317]:   <iothreads supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:   <os supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='firmware'/>
Apr 14 04:12:26 user nova-compute[71317]:     <loader supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>rom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pflash</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='readonly'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>yes</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='secure'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </loader>
Apr 14 04:12:26 user nova-compute[71317]:   </os>
Apr 14 04:12:26 user nova-compute[71317]:   <cpu>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-passthrough' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='hostPassthroughMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='maximum' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='maximumMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>on</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-model' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <vendor>Intel</vendor>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ss'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='vmx'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='pcid'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='hypervisor'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arat'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='umip'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='md-clear'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibpb'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibrs'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='erms'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='lm'/>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='custom' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>qemu64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>qemu32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>phenom</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>n270</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>kvm64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>kvm32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>coreduo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>core2duo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>athlon</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Westmere-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Westmere</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Snowridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Penryn</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G5</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G4</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G1</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Nehalem-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Nehalem</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Rome</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Milan</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Dhyana</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cooperlake</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Conroe</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>486</model>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:   </cpu>
Apr 14 04:12:26 user nova-compute[71317]:   <memoryBacking supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='sourceType'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>file</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>anonymous</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>memfd</value>
Apr 14 04:12:26 user nova-compute[71317]:     </enum>
Apr 14 04:12:26 user nova-compute[71317]:   </memoryBacking>
Apr 14 04:12:26 user nova-compute[71317]:   <devices>
Apr 14 04:12:26 user nova-compute[71317]:     <disk supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='diskDevice'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>disk</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cdrom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>floppy</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>lun</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='bus'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>ide</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>fdc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sata</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </disk>
Apr 14 04:12:26 user nova-compute[71317]:     <graphics supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sdl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vnc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>spice</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egl-headless</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </graphics>
Apr 14 04:12:26 user nova-compute[71317]:     <video supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='modelType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cirrus</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vmvga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>qxl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>none</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>bochs</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>ramfb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </video>
Apr 14 04:12:26 user nova-compute[71317]:     <hostdev supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='mode'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>subsystem</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='startupPolicy'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>default</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>mandatory</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>requisite</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>optional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='subsysType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pci</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='capsType'/>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='pciBackend'/>
Apr 14 04:12:26 user nova-compute[71317]:     </hostdev>
Apr 14 04:12:26 user nova-compute[71317]:     <rng supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>random</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egd</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>builtin</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </rng>
Apr 14 04:12:26 user nova-compute[71317]:     <filesystem supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='driverType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>path</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>handle</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtiofs</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </filesystem>
Apr 14 04:12:26 user nova-compute[71317]:     <tpm supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-tis</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-crb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>passthrough</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>emulator</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </tpm>
Apr 14 04:12:26 user nova-compute[71317]:   </devices>
Apr 14 04:12:26 user nova-compute[71317]:   <features>
Apr 14 04:12:26 user nova-compute[71317]:     <gic supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     <vmcoreinfo supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <genid supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backingStoreInput supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backup supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <sev supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:   </features>
Apr 14 04:12:26 user nova-compute[71317]: </domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:  {{(pid=71317) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=q35:
Apr 14 04:12:26 user nova-compute[71317]: <domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:   <path>/usr/bin/qemu-system-i386</path>
Apr 14 04:12:26 user nova-compute[71317]:   <domain>kvm</domain>
Apr 14 04:12:26 user nova-compute[71317]:   <machine>pc-q35-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:   <arch>i686</arch>
Apr 14 04:12:26 user nova-compute[71317]:   <vcpu max='288'/>
Apr 14 04:12:26 user nova-compute[71317]:   <iothreads supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:   <os supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='firmware'/>
Apr 14 04:12:26 user nova-compute[71317]:     <loader supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>rom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pflash</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='readonly'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>yes</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='secure'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </loader>
Apr 14 04:12:26 user nova-compute[71317]:   </os>
Apr 14 04:12:26 user nova-compute[71317]:   <cpu>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-passthrough' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='hostPassthroughMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='maximum' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='maximumMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>on</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-model' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <vendor>Intel</vendor>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ss'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='vmx'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='pcid'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='hypervisor'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arat'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='umip'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='md-clear'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibpb'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibrs'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='erms'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='lm'/>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='custom' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>qemu64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>qemu32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>phenom</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>n270</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>kvm64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>kvm32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>coreduo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>core2duo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>athlon</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Westmere-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Westmere</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Snowridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Penryn</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G5</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G4</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G1</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Nehalem-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Nehalem</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Rome</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Milan</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Dhyana</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cooperlake</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Conroe</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>486</model>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:   </cpu>
Apr 14 04:12:26 user nova-compute[71317]:   <memoryBacking supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='sourceType'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>file</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>anonymous</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>memfd</value>
Apr 14 04:12:26 user nova-compute[71317]:     </enum>
Apr 14 04:12:26 user nova-compute[71317]:   </memoryBacking>
Apr 14 04:12:26 user nova-compute[71317]:   <devices>
Apr 14 04:12:26 user nova-compute[71317]:     <disk supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='diskDevice'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>disk</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cdrom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>floppy</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>lun</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='bus'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>fdc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sata</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </disk>
Apr 14 04:12:26 user nova-compute[71317]:     <graphics supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sdl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vnc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>spice</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egl-headless</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </graphics>
Apr 14 04:12:26 user nova-compute[71317]:     <video supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='modelType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cirrus</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vmvga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>qxl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>none</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>bochs</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>ramfb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </video>
Apr 14 04:12:26 user nova-compute[71317]:     <hostdev supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='mode'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>subsystem</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='startupPolicy'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>default</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>mandatory</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>requisite</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>optional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='subsysType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pci</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='capsType'/>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='pciBackend'/>
Apr 14 04:12:26 user nova-compute[71317]:     </hostdev>
Apr 14 04:12:26 user nova-compute[71317]:     <rng supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>random</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egd</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>builtin</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </rng>
Apr 14 04:12:26 user nova-compute[71317]:     <filesystem supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='driverType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>path</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>handle</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtiofs</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </filesystem>
Apr 14 04:12:26 user nova-compute[71317]:     <tpm supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-tis</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-crb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>passthrough</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>emulator</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </tpm>
Apr 14 04:12:26 user nova-compute[71317]:   </devices>
Apr 14 04:12:26 user nova-compute[71317]:   <features>
Apr 14 04:12:26 user nova-compute[71317]:     <gic supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     <vmcoreinfo supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <genid supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backingStoreInput supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backup supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <sev supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:   </features>
Apr 14 04:12:26 user nova-compute[71317]: </domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:  {{(pid=71317) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=ubuntu:
Apr 14 04:12:26 user nova-compute[71317]: <domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:   <path>/usr/bin/qemu-system-i386</path>
Apr 14 04:12:26 user nova-compute[71317]:   <domain>kvm</domain>
Apr 14 04:12:26 user nova-compute[71317]:   <machine>pc-i440fx-jammy</machine>
Apr 14 04:12:26 user nova-compute[71317]:   <arch>i686</arch>
Apr 14 04:12:26 user nova-compute[71317]:   <vcpu max='255'/>
Apr 14 04:12:26 user nova-compute[71317]:   <iothreads supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:   <os supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='firmware'/>
Apr 14 04:12:26 user nova-compute[71317]:     <loader supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>rom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pflash</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='readonly'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>yes</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='secure'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </loader>
Apr 14 04:12:26 user nova-compute[71317]:   </os>
Apr 14 04:12:26 user nova-compute[71317]:   <cpu>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-passthrough' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='hostPassthroughMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='maximum' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='maximumMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>on</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-model' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <vendor>Intel</vendor>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ss'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='vmx'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='pcid'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='hypervisor'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arat'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='umip'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='md-clear'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibpb'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibrs'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='erms'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='lm'/>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='custom' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>qemu64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>qemu32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>phenom</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>n270</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>kvm64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>kvm32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>coreduo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>core2duo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>athlon</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Westmere-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Westmere</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Snowridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Penryn</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G5</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G4</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G1</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Nehalem-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Nehalem</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Rome</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Milan</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Dhyana</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cooperlake</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Conroe</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>486</model>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:   </cpu>
Apr 14 04:12:26 user nova-compute[71317]:   <memoryBacking supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='sourceType'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>file</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>anonymous</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>memfd</value>
Apr 14 04:12:26 user nova-compute[71317]:     </enum>
Apr 14 04:12:26 user nova-compute[71317]:   </memoryBacking>
Apr 14 04:12:26 user nova-compute[71317]:   <devices>
Apr 14 04:12:26 user nova-compute[71317]:     <disk supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='diskDevice'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>disk</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cdrom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>floppy</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>lun</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='bus'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>ide</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>fdc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sata</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </disk>
Apr 14 04:12:26 user nova-compute[71317]:     <graphics supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sdl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vnc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>spice</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egl-headless</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </graphics>
Apr 14 04:12:26 user nova-compute[71317]:     <video supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='modelType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cirrus</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vmvga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>qxl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>none</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>bochs</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>ramfb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </video>
Apr 14 04:12:26 user nova-compute[71317]:     <hostdev supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='mode'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>subsystem</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='startupPolicy'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>default</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>mandatory</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>requisite</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>optional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='subsysType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pci</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='capsType'/>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='pciBackend'/>
Apr 14 04:12:26 user nova-compute[71317]:     </hostdev>
Apr 14 04:12:26 user nova-compute[71317]:     <rng supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>random</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egd</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>builtin</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </rng>
Apr 14 04:12:26 user nova-compute[71317]:     <filesystem supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='driverType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>path</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>handle</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtiofs</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </filesystem>
Apr 14 04:12:26 user nova-compute[71317]:     <tpm supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-tis</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-crb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>passthrough</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>emulator</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </tpm>
Apr 14 04:12:26 user nova-compute[71317]:   </devices>
Apr 14 04:12:26 user nova-compute[71317]:   <features>
Apr 14 04:12:26 user nova-compute[71317]:     <gic supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     <vmcoreinfo supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <genid supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backingStoreInput supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backup supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <sev supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:   </features>
Apr 14 04:12:26 user nova-compute[71317]: </domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:  {{(pid=71317) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=ubuntu-q35:
Apr 14 04:12:26 user nova-compute[71317]: <domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:   <path>/usr/bin/qemu-system-i386</path>
Apr 14 04:12:26 user nova-compute[71317]:   <domain>kvm</domain>
Apr 14 04:12:26 user nova-compute[71317]:   <machine>pc-q35-jammy</machine>
Apr 14 04:12:26 user nova-compute[71317]:   <arch>i686</arch>
Apr 14 04:12:26 user nova-compute[71317]:   <vcpu max='288'/>
Apr 14 04:12:26 user nova-compute[71317]:   <iothreads supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:   <os supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='firmware'/>
Apr 14 04:12:26 user nova-compute[71317]:     <loader supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>rom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pflash</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='readonly'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>yes</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='secure'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </loader>
Apr 14 04:12:26 user nova-compute[71317]:   </os>
Apr 14 04:12:26 user nova-compute[71317]:   <cpu>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-passthrough' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='hostPassthroughMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='maximum' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='maximumMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>on</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-model' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <vendor>Intel</vendor>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ss'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='vmx'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='pcid'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='hypervisor'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arat'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='umip'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='md-clear'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibpb'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibrs'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='erms'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='lm'/>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='custom' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>qemu64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>qemu32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>phenom</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>n270</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>kvm64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>kvm32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>coreduo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>core2duo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>athlon</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Westmere-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Westmere</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Snowridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Penryn</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G5</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G4</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G1</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Nehalem-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Nehalem</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Rome</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Milan</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Dhyana</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cooperlake</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Conroe</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>486</model>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:   </cpu>
Apr 14 04:12:26 user nova-compute[71317]:   <memoryBacking supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='sourceType'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>file</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>anonymous</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>memfd</value>
Apr 14 04:12:26 user nova-compute[71317]:     </enum>
Apr 14 04:12:26 user nova-compute[71317]:   </memoryBacking>
Apr 14 04:12:26 user nova-compute[71317]:   <devices>
Apr 14 04:12:26 user nova-compute[71317]:     <disk supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='diskDevice'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>disk</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cdrom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>floppy</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>lun</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='bus'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>fdc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sata</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </disk>
Apr 14 04:12:26 user nova-compute[71317]:     <graphics supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sdl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vnc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>spice</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egl-headless</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </graphics>
Apr 14 04:12:26 user nova-compute[71317]:     <video supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='modelType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cirrus</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vmvga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>qxl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>none</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>bochs</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>ramfb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </video>
Apr 14 04:12:26 user nova-compute[71317]:     <hostdev supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='mode'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>subsystem</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='startupPolicy'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>default</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>mandatory</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>requisite</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>optional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='subsysType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pci</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='capsType'/>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='pciBackend'/>
Apr 14 04:12:26 user nova-compute[71317]:     </hostdev>
Apr 14 04:12:26 user nova-compute[71317]:     <rng supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>random</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egd</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>builtin</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </rng>
Apr 14 04:12:26 user nova-compute[71317]:     <filesystem supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='driverType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>path</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>handle</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtiofs</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </filesystem>
Apr 14 04:12:26 user nova-compute[71317]:     <tpm supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-tis</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-crb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>passthrough</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>emulator</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </tpm>
Apr 14 04:12:26 user nova-compute[71317]:   </devices>
Apr 14 04:12:26 user nova-compute[71317]:   <features>
Apr 14 04:12:26 user nova-compute[71317]:     <gic supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     <vmcoreinfo supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <genid supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backingStoreInput supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backup supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <sev supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:   </features>
Apr 14 04:12:26 user nova-compute[71317]: </domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:  {{(pid=71317) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for m68k via machine types: {'virt', None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch m68k / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-m68k' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch m68k / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-m68k' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for microblaze via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch microblaze / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-microblaze' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for microblazeel via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch microblazeel / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-microblazeel' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for mips via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch mips / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for mipsel via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch mipsel / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mipsel' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for mips64 via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch mips64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips64' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for mips64el via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch mips64el / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips64el' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for ppc via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch ppc / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for ppc64 via machine types: {'powernv', None, 'pseries'} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type powernv: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type pseries: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for ppc64le via machine types: {'powernv', 'pseries'} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch ppc64le / virt_type kvm / machine_type powernv: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch ppc64le / virt_type kvm / machine_type pseries: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for riscv32 via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch riscv32 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-riscv32' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for riscv64 via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch riscv64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-riscv64' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for s390x via machine types: {'s390-ccw-virtio'} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch s390x / virt_type kvm / machine_type s390-ccw-virtio: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-s390x' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for sh4 via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch sh4 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sh4' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for sh4eb via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch sh4eb / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sh4eb' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for sparc via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch sparc / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sparc' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for sparc64 via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch sparc64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sparc64' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for x86_64 via machine types: {'pc', 'q35', 'ubuntu', 'ubuntu-q35'} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=pc:
Apr 14 04:12:26 user nova-compute[71317]: <domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:   <path>/usr/bin/qemu-system-x86_64</path>
Apr 14 04:12:26 user nova-compute[71317]:   <domain>kvm</domain>
Apr 14 04:12:26 user nova-compute[71317]:   <machine>pc-i440fx-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:   <arch>x86_64</arch>
Apr 14 04:12:26 user nova-compute[71317]:   <vcpu max='255'/>
Apr 14 04:12:26 user nova-compute[71317]:   <iothreads supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:   <os supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='firmware'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>efi</value>
Apr 14 04:12:26 user nova-compute[71317]:     </enum>
Apr 14 04:12:26 user nova-compute[71317]:     <loader supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>rom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pflash</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='readonly'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>yes</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='secure'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </loader>
Apr 14 04:12:26 user nova-compute[71317]:   </os>
Apr 14 04:12:26 user nova-compute[71317]:   <cpu>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-passthrough' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='hostPassthroughMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>on</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='maximum' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='maximumMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>on</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-model' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <vendor>Intel</vendor>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ss'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='vmx'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='pcid'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='hypervisor'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arat'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='umip'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='md-clear'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='invtsc'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibpb'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibrs'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='rsba'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='pschange-mc-no'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='erms'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='custom' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>qemu64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>qemu32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>phenom</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>n270</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>kvm64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>kvm32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>coreduo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>core2duo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>athlon</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Westmere-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Westmere</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Snowridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Penryn</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G5</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G4</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Opteron_G1</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Nehalem-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Nehalem</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Rome</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Milan</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Dhyana</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cooperlake</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Conroe</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>486</model>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:   </cpu>
Apr 14 04:12:26 user nova-compute[71317]:   <memoryBacking supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='sourceType'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>file</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>anonymous</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>memfd</value>
Apr 14 04:12:26 user nova-compute[71317]:     </enum>
Apr 14 04:12:26 user nova-compute[71317]:   </memoryBacking>
Apr 14 04:12:26 user nova-compute[71317]:   <devices>
Apr 14 04:12:26 user nova-compute[71317]:     <disk supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='diskDevice'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>disk</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cdrom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>floppy</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>lun</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='bus'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>ide</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>fdc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sata</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </disk>
Apr 14 04:12:26 user nova-compute[71317]:     <graphics supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sdl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vnc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>spice</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egl-headless</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </graphics>
Apr 14 04:12:26 user nova-compute[71317]:     <video supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='modelType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cirrus</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vmvga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>qxl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>none</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>bochs</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>ramfb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </video>
Apr 14 04:12:26 user nova-compute[71317]:     <hostdev supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='mode'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>subsystem</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='startupPolicy'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>default</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>mandatory</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>requisite</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>optional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='subsysType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pci</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='capsType'/>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='pciBackend'/>
Apr 14 04:12:26 user nova-compute[71317]:     </hostdev>
Apr 14 04:12:26 user nova-compute[71317]:     <rng supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>random</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egd</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>builtin</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </rng>
Apr 14 04:12:26 user nova-compute[71317]:     <filesystem supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='driverType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>path</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>handle</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtiofs</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </filesystem>
Apr 14 04:12:26 user nova-compute[71317]:     <tpm supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-tis</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-crb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>passthrough</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>emulator</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </tpm>
Apr 14 04:12:26 user nova-compute[71317]:   </devices>
Apr 14 04:12:26 user nova-compute[71317]:   <features>
Apr 14 04:12:26 user nova-compute[71317]:     <gic supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     <vmcoreinfo supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <genid supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backingStoreInput supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backup supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <sev supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:   </features>
Apr 14 04:12:26 user nova-compute[71317]: </domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:  {{(pid=71317) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=q35:
Apr 14 04:12:26 user nova-compute[71317]: <domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:   <path>/usr/bin/qemu-system-x86_64</path>
Apr 14 04:12:26 user nova-compute[71317]:   <domain>kvm</domain>
Apr 14 04:12:26 user nova-compute[71317]:   <machine>pc-q35-6.2</machine>
Apr 14 04:12:26 user nova-compute[71317]:   <arch>x86_64</arch>
Apr 14 04:12:26 user nova-compute[71317]:   <vcpu max='288'/>
Apr 14 04:12:26 user nova-compute[71317]:   <iothreads supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:   <os supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='firmware'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>efi</value>
Apr 14 04:12:26 user nova-compute[71317]:     </enum>
Apr 14 04:12:26 user nova-compute[71317]:     <loader supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE_4M.ms.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE_4M.secboot.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>rom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pflash</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='readonly'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>yes</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='secure'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>yes</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </loader>
Apr 14 04:12:26 user nova-compute[71317]:   </os>
Apr 14 04:12:26 user nova-compute[71317]:   <cpu>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-passthrough' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='hostPassthroughMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>on</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='maximum' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='maximumMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>on</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-model' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <vendor>Intel</vendor>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ss'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='vmx'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='pcid'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='hypervisor'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arat'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='umip'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='md-clear'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='invtsc'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibpb'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibrs'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='rsba'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='pschange-mc-no'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='erms'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='custom' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>qemu64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>qemu32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>phenom</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>n270</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>kvm64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>kvm32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>coreduo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>core2duo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>athlon</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Westmere-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Westmere</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Snowridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Penryn</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G5</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G4</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Opteron_G1</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Nehalem-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Nehalem</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Rome</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Milan</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Dhyana</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cooperlake</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Conroe</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>486</model>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:   </cpu>
Apr 14 04:12:26 user nova-compute[71317]:   <memoryBacking supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='sourceType'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>file</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>anonymous</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>memfd</value>
Apr 14 04:12:26 user nova-compute[71317]:     </enum>
Apr 14 04:12:26 user nova-compute[71317]:   </memoryBacking>
Apr 14 04:12:26 user nova-compute[71317]:   <devices>
Apr 14 04:12:26 user nova-compute[71317]:     <disk supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='diskDevice'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>disk</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cdrom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>floppy</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>lun</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='bus'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>fdc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sata</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </disk>
Apr 14 04:12:26 user nova-compute[71317]:     <graphics supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sdl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vnc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>spice</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egl-headless</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </graphics>
Apr 14 04:12:26 user nova-compute[71317]:     <video supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='modelType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cirrus</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vmvga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>qxl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>none</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>bochs</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>ramfb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </video>
Apr 14 04:12:26 user nova-compute[71317]:     <hostdev supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='mode'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>subsystem</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='startupPolicy'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>default</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>mandatory</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>requisite</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>optional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='subsysType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pci</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='capsType'/>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='pciBackend'/>
Apr 14 04:12:26 user nova-compute[71317]:     </hostdev>
Apr 14 04:12:26 user nova-compute[71317]:     <rng supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>random</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egd</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>builtin</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </rng>
Apr 14 04:12:26 user nova-compute[71317]:     <filesystem supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='driverType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>path</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>handle</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtiofs</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </filesystem>
Apr 14 04:12:26 user nova-compute[71317]:     <tpm supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-tis</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-crb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>passthrough</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>emulator</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </tpm>
Apr 14 04:12:26 user nova-compute[71317]:   </devices>
Apr 14 04:12:26 user nova-compute[71317]:   <features>
Apr 14 04:12:26 user nova-compute[71317]:     <gic supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     <vmcoreinfo supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <genid supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backingStoreInput supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backup supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <sev supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:   </features>
Apr 14 04:12:26 user nova-compute[71317]: </domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:  {{(pid=71317) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=ubuntu:
Apr 14 04:12:26 user nova-compute[71317]: <domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:   <path>/usr/bin/qemu-system-x86_64</path>
Apr 14 04:12:26 user nova-compute[71317]:   <domain>kvm</domain>
Apr 14 04:12:26 user nova-compute[71317]:   <machine>pc-i440fx-jammy</machine>
Apr 14 04:12:26 user nova-compute[71317]:   <arch>x86_64</arch>
Apr 14 04:12:26 user nova-compute[71317]:   <vcpu max='255'/>
Apr 14 04:12:26 user nova-compute[71317]:   <iothreads supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:   <os supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='firmware'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>efi</value>
Apr 14 04:12:26 user nova-compute[71317]:     </enum>
Apr 14 04:12:26 user nova-compute[71317]:     <loader supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>rom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pflash</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='readonly'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>yes</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='secure'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </loader>
Apr 14 04:12:26 user nova-compute[71317]:   </os>
Apr 14 04:12:26 user nova-compute[71317]:   <cpu>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-passthrough' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='hostPassthroughMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>on</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='maximum' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='maximumMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>on</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-model' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <vendor>Intel</vendor>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ss'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='vmx'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='pcid'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='hypervisor'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arat'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='umip'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='md-clear'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='invtsc'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibpb'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibrs'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='rsba'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='pschange-mc-no'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='erms'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='custom' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>qemu64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>qemu32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>phenom</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>n270</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>kvm64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>kvm32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>coreduo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>core2duo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>athlon</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Westmere-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Westmere</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Snowridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Penryn</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G5</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G4</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Opteron_G1</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Nehalem-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Nehalem</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Rome</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Milan</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Dhyana</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cooperlake</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Conroe</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>486</model>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:   </cpu>
Apr 14 04:12:26 user nova-compute[71317]:   <memoryBacking supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='sourceType'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>file</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>anonymous</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>memfd</value>
Apr 14 04:12:26 user nova-compute[71317]:     </enum>
Apr 14 04:12:26 user nova-compute[71317]:   </memoryBacking>
Apr 14 04:12:26 user nova-compute[71317]:   <devices>
Apr 14 04:12:26 user nova-compute[71317]:     <disk supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='diskDevice'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>disk</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cdrom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>floppy</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>lun</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='bus'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>ide</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>fdc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sata</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </disk>
Apr 14 04:12:26 user nova-compute[71317]:     <graphics supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sdl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vnc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>spice</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egl-headless</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </graphics>
Apr 14 04:12:26 user nova-compute[71317]:     <video supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='modelType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cirrus</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vmvga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>qxl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>none</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>bochs</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>ramfb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </video>
Apr 14 04:12:26 user nova-compute[71317]:     <hostdev supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='mode'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>subsystem</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='startupPolicy'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>default</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>mandatory</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>requisite</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>optional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='subsysType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pci</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='capsType'/>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='pciBackend'/>
Apr 14 04:12:26 user nova-compute[71317]:     </hostdev>
Apr 14 04:12:26 user nova-compute[71317]:     <rng supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>random</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egd</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>builtin</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </rng>
Apr 14 04:12:26 user nova-compute[71317]:     <filesystem supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='driverType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>path</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>handle</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtiofs</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </filesystem>
Apr 14 04:12:26 user nova-compute[71317]:     <tpm supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-tis</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-crb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>passthrough</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>emulator</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </tpm>
Apr 14 04:12:26 user nova-compute[71317]:   </devices>
Apr 14 04:12:26 user nova-compute[71317]:   <features>
Apr 14 04:12:26 user nova-compute[71317]:     <gic supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     <vmcoreinfo supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <genid supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backingStoreInput supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backup supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <sev supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:   </features>
Apr 14 04:12:26 user nova-compute[71317]: </domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:  {{(pid=71317) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=ubuntu-q35:
Apr 14 04:12:26 user nova-compute[71317]: <domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:   <path>/usr/bin/qemu-system-x86_64</path>
Apr 14 04:12:26 user nova-compute[71317]:   <domain>kvm</domain>
Apr 14 04:12:26 user nova-compute[71317]:   <machine>pc-q35-jammy</machine>
Apr 14 04:12:26 user nova-compute[71317]:   <arch>x86_64</arch>
Apr 14 04:12:26 user nova-compute[71317]:   <vcpu max='288'/>
Apr 14 04:12:26 user nova-compute[71317]:   <iothreads supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:   <os supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='firmware'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>efi</value>
Apr 14 04:12:26 user nova-compute[71317]:     </enum>
Apr 14 04:12:26 user nova-compute[71317]:     <loader supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE_4M.ms.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE_4M.secboot.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>rom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pflash</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='readonly'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>yes</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='secure'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>yes</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>no</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </loader>
Apr 14 04:12:26 user nova-compute[71317]:   </os>
Apr 14 04:12:26 user nova-compute[71317]:   <cpu>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-passthrough' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='hostPassthroughMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>on</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='maximum' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='maximumMigratable'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>on</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>off</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='host-model' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <vendor>Intel</vendor>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ss'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='vmx'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='pcid'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='hypervisor'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arat'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='umip'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='md-clear'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='invtsc'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibpb'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='ibrs'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-stibp'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='rsba'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='require' name='pschange-mc-no'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='erms'/>
Apr 14 04:12:26 user nova-compute[71317]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:     <mode name='custom' supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>qemu64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>qemu32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>phenom</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>pentium</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>n270</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>kvm64</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>kvm32</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>coreduo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>core2duo</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>athlon</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Westmere-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Westmere</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Snowridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Skylake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>SandyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Penryn</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G5</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G4</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G3</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Opteron_G2</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Opteron_G1</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Nehalem-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Nehalem</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>IvyBridge</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Icelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Haswell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Rome</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-Milan</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>EPYC</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Dhyana</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cooperlake</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>Conroe</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='no'>Broadwell</model>
Apr 14 04:12:26 user nova-compute[71317]:       <model usable='yes'>486</model>
Apr 14 04:12:26 user nova-compute[71317]:     </mode>
Apr 14 04:12:26 user nova-compute[71317]:   </cpu>
Apr 14 04:12:26 user nova-compute[71317]:   <memoryBacking supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:     <enum name='sourceType'>
Apr 14 04:12:26 user nova-compute[71317]:       <value>file</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>anonymous</value>
Apr 14 04:12:26 user nova-compute[71317]:       <value>memfd</value>
Apr 14 04:12:26 user nova-compute[71317]:     </enum>
Apr 14 04:12:26 user nova-compute[71317]:   </memoryBacking>
Apr 14 04:12:26 user nova-compute[71317]:   <devices>
Apr 14 04:12:26 user nova-compute[71317]:     <disk supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='diskDevice'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>disk</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cdrom</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>floppy</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>lun</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='bus'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>fdc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sata</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </disk>
Apr 14 04:12:26 user nova-compute[71317]:     <graphics supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='type'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>sdl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vnc</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>spice</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egl-headless</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </graphics>
Apr 14 04:12:26 user nova-compute[71317]:     <video supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='modelType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>cirrus</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>vmvga</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>qxl</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>none</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>bochs</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>ramfb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </video>
Apr 14 04:12:26 user nova-compute[71317]:     <hostdev supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='mode'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>subsystem</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='startupPolicy'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>default</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>mandatory</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>requisite</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>optional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='subsysType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>usb</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>pci</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>scsi</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='capsType'/>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='pciBackend'/>
Apr 14 04:12:26 user nova-compute[71317]:     </hostdev>
Apr 14 04:12:26 user nova-compute[71317]:     <rng supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtio-non-transitional</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>random</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>egd</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>builtin</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </rng>
Apr 14 04:12:26 user nova-compute[71317]:     <filesystem supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='driverType'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>path</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>handle</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>virtiofs</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </filesystem>
Apr 14 04:12:26 user nova-compute[71317]:     <tpm supported='yes'>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='model'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-tis</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>tpm-crb</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:       <enum name='backendModel'>
Apr 14 04:12:26 user nova-compute[71317]:         <value>passthrough</value>
Apr 14 04:12:26 user nova-compute[71317]:         <value>emulator</value>
Apr 14 04:12:26 user nova-compute[71317]:       </enum>
Apr 14 04:12:26 user nova-compute[71317]:     </tpm>
Apr 14 04:12:26 user nova-compute[71317]:   </devices>
Apr 14 04:12:26 user nova-compute[71317]:   <features>
Apr 14 04:12:26 user nova-compute[71317]:     <gic supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:     <vmcoreinfo supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <genid supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backingStoreInput supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <backup supported='yes'/>
Apr 14 04:12:26 user nova-compute[71317]:     <sev supported='no'/>
Apr 14 04:12:26 user nova-compute[71317]:   </features>
Apr 14 04:12:26 user nova-compute[71317]: </domainCapabilities>
Apr 14 04:12:26 user nova-compute[71317]:  {{(pid=71317) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for xtensa via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch xtensa / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-xtensa' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Getting domain capabilities for xtensaeb via machine types: {None} {{(pid=71317) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Error from libvirt when retrieving domain capabilities for arch xtensaeb / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-xtensaeb' on this host {{(pid=71317) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Checking secure boot support for host arch (x86_64) {{(pid=71317) supports_secure_boot /opt/stack/nova/nova/virt/libvirt/host.py:1750}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Checking secure boot support for host arch (x86_64) {{(pid=71317) supports_secure_boot /opt/stack/nova/nova/virt/libvirt/host.py:1750}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Checking secure boot support for host arch (x86_64) {{(pid=71317) supports_secure_boot /opt/stack/nova/nova/virt/libvirt/host.py:1750}}
Apr 14 04:12:26 user nova-compute[71317]: INFO nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Secure Boot support detected
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] cpu compare xml: <cpu match="exact">
Apr 14 04:12:26 user nova-compute[71317]:   <model>Nehalem</model>
Apr 14 04:12:26 user nova-compute[71317]: </cpu>
Apr 14 04:12:26 user nova-compute[71317]:  {{(pid=71317) _compare_cpu /opt/stack/nova/nova/virt/libvirt/driver.py:9996}}
Apr 14 04:12:26 user nova-compute[71317]: INFO nova.virt.node [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Generated node identity e35a99e4-b1ea-4792-a9ff-f93045b2018b
Apr 14 04:12:26 user nova-compute[71317]: INFO nova.virt.node [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Wrote node identity e35a99e4-b1ea-4792-a9ff-f93045b2018b to /opt/stack/data/nova/compute_id
Apr 14 04:12:26 user nova-compute[71317]: WARNING nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Compute nodes ['e35a99e4-b1ea-4792-a9ff-f93045b2018b'] for host user were not found in the database. If this is the first time this service is starting on this host, then you can ignore this warning.
Apr 14 04:12:26 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Looking for unclaimed instances stuck in BUILDING status for nodes managed by this host
Apr 14 04:12:26 user nova-compute[71317]: WARNING nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] No compute node record found for host user. If this is the first time this service is starting on this host, then you can ignore this warning.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host user could not be found.
Apr 14 04:12:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:12:26 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:12:26 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Hypervisor/Node resource view: name=user free_ram=10818MB free_disk=27.182003021240234GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:12:26 user nova-compute[71317]: WARNING nova.compute.resource_tracker [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] No compute node record for user:e35a99e4-b1ea-4792-a9ff-f93045b2018b: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host e35a99e4-b1ea-4792-a9ff-f93045b2018b could not be found.
Apr 14 04:12:26 user nova-compute[71317]: INFO nova.compute.resource_tracker [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Compute node record created for user:user with uuid: e35a99e4-b1ea-4792-a9ff-f93045b2018b
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:12:26 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:12:27 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [req-ac60b8d4-4309-40c2-beea-a7e6a4793e71] Created resource provider record via placement API for resource provider with UUID e35a99e4-b1ea-4792-a9ff-f93045b2018b and name user.
Apr 14 04:12:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] /sys/module/kvm_amd/parameters/sev does not exist {{(pid=71317) _kernel_supports_amd_sev /opt/stack/nova/nova/virt/libvirt/host.py:1766}}
Apr 14 04:12:27 user nova-compute[71317]: INFO nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] kernel doesn't support AMD SEV
Apr 14 04:12:27 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Updating inventory in ProviderTree for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b with inventory: {'MEMORY_MB': {'total': 16023, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 512}, 'VCPU': {'total': 12, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0, 'reserved': 0}, 'DISK_GB': {'total': 40, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}} {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 04:12:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:12:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Libvirt baseline CPU <cpu>
Apr 14 04:12:27 user nova-compute[71317]:   <arch>x86_64</arch>
Apr 14 04:12:27 user nova-compute[71317]:   <model>Nehalem</model>
Apr 14 04:12:27 user nova-compute[71317]:   <vendor>Intel</vendor>
Apr 14 04:12:27 user nova-compute[71317]:   <topology sockets="6" cores="1" threads="1"/>
Apr 14 04:12:27 user nova-compute[71317]: </cpu>
Apr 14 04:12:27 user nova-compute[71317]:  {{(pid=71317) _get_guest_baseline_cpu_features /opt/stack/nova/nova/virt/libvirt/driver.py:12486}}
Apr 14 04:12:27 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Updated inventory for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b with generation 0 in Placement from set_inventory_for_provider using data: {'MEMORY_MB': {'total': 16023, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 512}, 'VCPU': {'total': 12, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0, 'reserved': 0}, 'DISK_GB': {'total': 40, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:957}}
Apr 14 04:12:27 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Updating resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b generation from 0 to 1 during operation: update_inventory {{(pid=71317) _update_generation /opt/stack/nova/nova/compute/provider_tree.py:164}}
Apr 14 04:12:27 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Updating inventory in ProviderTree for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b with inventory: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 04:12:27 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Updating resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b generation from 1 to 2 during operation: update_traits {{(pid=71317) _update_generation /opt/stack/nova/nova/compute/provider_tree.py:164}}
Apr 14 04:12:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:12:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.673s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:12:27 user nova-compute[71317]: DEBUG nova.service [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Creating RPC server for service compute {{(pid=71317) start /opt/stack/nova/nova/service.py:182}}
Apr 14 04:12:27 user nova-compute[71317]: DEBUG nova.service [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Join ServiceGroup membership for this service compute {{(pid=71317) start /opt/stack/nova/nova/service.py:199}}
Apr 14 04:12:27 user nova-compute[71317]: DEBUG nova.servicegroup.drivers.db [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] DB_Driver: join new ServiceGroup member user to the compute group, service = <Service: host=user, binary=nova-compute, manager_class_name=nova.compute.manager.ComputeManager> {{(pid=71317) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Didn't find any instances for network info cache update. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_power_states {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:13:22 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:13:23 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:13:23 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:13:23 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=10205MB free_disk=27.09576416015625GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:13:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:13:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:13:23 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:13:23 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:13:23 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:13:23 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:13:23 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:13:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.171s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:13:23 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_running_deleted_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Didn't find any instances for network info cache update. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:14:18 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:14:19 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:14:19 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:14:19 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=10156MB free_disk=27.143051147460938GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:14:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:14:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:14:19 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:14:19 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:14:19 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:14:19 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:14:19 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:14:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.132s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:14:19 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:14:19 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:14:20 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:15:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:15:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:15:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Didn't find any instances for network info cache update. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:15:19 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:15:19 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:15:19 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=10150MB free_disk=26.918006896972656GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:15:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.142s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:15:21 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:15:21 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:16:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:16:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:16:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:16:19 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:16:20 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:16:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:16:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:16:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Didn't find any instances for network info cache update. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 04:16:20 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:16:21 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:21 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:21 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=9431MB free_disk=26.940887451171875GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:16:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.144s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "0ff97860-2441-437c-b118-6089c3cc4160" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "0ff97860-2441-437c-b118-6089c3cc4160" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:16:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:16:22 user nova-compute[71317]: INFO nova.compute.claims [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Claim successful on node user
Apr 14 04:16:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:16:22 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:16:22 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:16:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.365s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:16:23 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:16:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:16:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:16:23 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:16:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:16:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:16:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:16:23 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Creating image(s)
Apr 14 04:16:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "/opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "/opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "/opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.004s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d.part --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:24 user nova-compute[71317]: DEBUG nova.policy [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '6e422970cd8340bf996a135088094f51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '280aa55745a749b7baa134334d25c9c3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:16:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d.part --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:24 user nova-compute[71317]: DEBUG nova.virt.images [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] 5c55a1da-f6f8-4442-939b-f0cb72ea4327 was qcow2, converting to raw {{(pid=71317) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 04:16:24 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:16:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d.part /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d.converted {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d.part /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d.converted" returned: 0 in 0.322s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d.converted --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d.converted --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.427s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:24 user nova-compute[71317]: INFO oslo.privsep.daemon [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'nova.privsep.sys_admin_pctxt', '--privsep_sock_path', '/tmp/tmpsai37e_f/privsep.sock']
Apr 14 04:16:24 user sudo[80036]:    stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context nova.privsep.sys_admin_pctxt --privsep_sock_path /tmp/tmpsai37e_f/privsep.sock
Apr 14 04:16:24 user sudo[80036]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Apr 14 04:16:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "9b532a44-20b7-4590-babe-684c49598a62" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "9b532a44-20b7-4590-babe-684c49598a62" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:16:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:25 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:16:25 user nova-compute[71317]: INFO nova.compute.claims [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Claim successful on node user
Apr 14 04:16:25 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:16:25 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:16:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.320s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:16:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:16:25 user nova-compute[71317]: DEBUG nova.network.neutron [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:16:25 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:16:26 user nova-compute[71317]: DEBUG nova.compute.manager [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:16:26 user nova-compute[71317]: DEBUG nova.compute.manager [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:16:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:16:26 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Creating image(s)
Apr 14 04:16:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "/opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "/opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "/opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:26 user nova-compute[71317]: DEBUG nova.policy [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ea68fb139b004f748cf1bcfb8d675a2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bae0fb4af555422a847620627dffeae8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:16:26 user sudo[80036]: pam_unix(sudo:session): session closed for user root
Apr 14 04:16:26 user nova-compute[71317]: INFO oslo.privsep.daemon [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Spawned new privsep daemon via rootwrap
Apr 14 04:16:26 user nova-compute[71317]: INFO oslo.privsep.daemon [-] privsep daemon starting
Apr 14 04:16:26 user nova-compute[71317]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Apr 14 04:16:26 user nova-compute[71317]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/none
Apr 14 04:16:26 user nova-compute[71317]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 80043
Apr 14 04:16:26 user nova-compute[71317]: WARNING oslo_privsep.priv_context [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] privsep daemon already running
Apr 14 04:16:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.125s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.125s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.123s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk 1073741824" returned: 0 in 0.042s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.170s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG nova.network.neutron [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Successfully created port: 22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.175s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.130s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Checking if we can resize image /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk 1073741824" returned: 0 in 0.042s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.185s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Cannot resize image /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG nova.objects.instance [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lazy-loading 'migration_context' on Instance uuid 9b532a44-20b7-4590-babe-684c49598a62 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Ensure instance console log exists: /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Checking if we can resize image /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Cannot resize image /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG nova.objects.instance [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lazy-loading 'migration_context' on Instance uuid 0ff97860-2441-437c-b118-6089c3cc4160 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Ensure instance console log exists: /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:28 user nova-compute[71317]: DEBUG nova.network.neutron [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Successfully created port: c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:16:30 user nova-compute[71317]: DEBUG nova.network.neutron [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Successfully updated port: c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:16:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "refresh_cache-9b532a44-20b7-4590-babe-684c49598a62" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquired lock "refresh_cache-9b532a44-20b7-4590-babe-684c49598a62" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:30 user nova-compute[71317]: DEBUG nova.network.neutron [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:16:30 user nova-compute[71317]: DEBUG nova.network.neutron [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:16:30 user nova-compute[71317]: DEBUG nova.compute.manager [req-ac4ff65a-557a-4c8d-a64e-96e319749867 req-45263051-7492-400d-9f96-5b461fdcb6d7 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Received event network-changed-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:30 user nova-compute[71317]: DEBUG nova.compute.manager [req-ac4ff65a-557a-4c8d-a64e-96e319749867 req-45263051-7492-400d-9f96-5b461fdcb6d7 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Refreshing instance network info cache due to event network-changed-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:16:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ac4ff65a-557a-4c8d-a64e-96e319749867 req-45263051-7492-400d-9f96-5b461fdcb6d7 service nova] Acquiring lock "refresh_cache-9b532a44-20b7-4590-babe-684c49598a62" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.network.neutron [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Successfully updated port: 22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "refresh_cache-0ff97860-2441-437c-b118-6089c3cc4160" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquired lock "refresh_cache-0ff97860-2441-437c-b118-6089c3cc4160" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.network.neutron [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.network.neutron [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.compute.manager [req-417129e4-3423-4518-a8f7-a56cbac38a48 req-72fa39a9-c49b-489f-bda4-40b14b1491fd service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Received event network-changed-22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.compute.manager [req-417129e4-3423-4518-a8f7-a56cbac38a48 req-72fa39a9-c49b-489f-bda4-40b14b1491fd service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Refreshing instance network info cache due to event network-changed-22212b8b-6656-4d88-ad5c-e0d8f657062f. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-417129e4-3423-4518-a8f7-a56cbac38a48 req-72fa39a9-c49b-489f-bda4-40b14b1491fd service nova] Acquiring lock "refresh_cache-0ff97860-2441-437c-b118-6089c3cc4160" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.network.neutron [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Updating instance_info_cache with network_info: [{"id": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "address": "fa:16:3e:47:9f:40", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc45f9c53-8d", "ovs_interfaceid": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Releasing lock "refresh_cache-9b532a44-20b7-4590-babe-684c49598a62" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Instance network_info: |[{"id": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "address": "fa:16:3e:47:9f:40", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc45f9c53-8d", "ovs_interfaceid": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ac4ff65a-557a-4c8d-a64e-96e319749867 req-45263051-7492-400d-9f96-5b461fdcb6d7 service nova] Acquired lock "refresh_cache-9b532a44-20b7-4590-babe-684c49598a62" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.network.neutron [req-ac4ff65a-557a-4c8d-a64e-96e319749867 req-45263051-7492-400d-9f96-5b461fdcb6d7 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Refreshing network info cache for port c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Start _get_guest_xml network_info=[{"id": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "address": "fa:16:3e:47:9f:40", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc45f9c53-8d", "ovs_interfaceid": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:16:31 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:31 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1687697238',display_name='tempest-ServersNegativeTestJSON-server-1687697238',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1687697238',id=2,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bae0fb4af555422a847620627dffeae8',ramdisk_id='',reservation_id='r-70d12ce1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-406789078',owner_user_name='tempest-ServersNegativeTestJSON-406789078-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:26Z,user_data=None,user_id='ea68fb139b004f748cf1bcfb8d675a2e',uuid=9b532a44-20b7-4590-babe-684c49598a62,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "address": "fa:16:3e:47:9f:40", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc45f9c53-8d", "ovs_interfaceid": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Converting VIF {"id": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "address": "fa:16:3e:47:9f:40", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc45f9c53-8d", "ovs_interfaceid": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:47:9f:40,bridge_name='br-int',has_traffic_filtering=True,id=c45f9c53-8d5c-4b47-8e0a-ebb292cefb06,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc45f9c53-8d') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.objects.instance [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lazy-loading 'pci_devices' on Instance uuid 9b532a44-20b7-4590-babe-684c49598a62 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:16:31 user nova-compute[71317]:   <uuid>9b532a44-20b7-4590-babe-684c49598a62</uuid>
Apr 14 04:16:31 user nova-compute[71317]:   <name>instance-00000002</name>
Apr 14 04:16:31 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:16:31 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:16:31 user nova-compute[71317]:   <metadata>
Apr 14 04:16:31 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:16:31 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:16:31 user nova-compute[71317]:       <nova:name>tempest-ServersNegativeTestJSON-server-1687697238</nova:name>
Apr 14 04:16:31 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:16:31</nova:creationTime>
Apr 14 04:16:31 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:16:31 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:16:31 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:16:31 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:16:31 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:16:31 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:16:31 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:16:31 user nova-compute[71317]:       <nova:owner>
Apr 14 04:16:31 user nova-compute[71317]:         <nova:user uuid="ea68fb139b004f748cf1bcfb8d675a2e">tempest-ServersNegativeTestJSON-406789078-project-member</nova:user>
Apr 14 04:16:31 user nova-compute[71317]:         <nova:project uuid="bae0fb4af555422a847620627dffeae8">tempest-ServersNegativeTestJSON-406789078</nova:project>
Apr 14 04:16:31 user nova-compute[71317]:       </nova:owner>
Apr 14 04:16:31 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:16:31 user nova-compute[71317]:       <nova:ports>
Apr 14 04:16:31 user nova-compute[71317]:         <nova:port uuid="c45f9c53-8d5c-4b47-8e0a-ebb292cefb06">
Apr 14 04:16:31 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 14 04:16:31 user nova-compute[71317]:         </nova:port>
Apr 14 04:16:31 user nova-compute[71317]:       </nova:ports>
Apr 14 04:16:31 user nova-compute[71317]:     </nova:instance>
Apr 14 04:16:31 user nova-compute[71317]:   </metadata>
Apr 14 04:16:31 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:16:31 user nova-compute[71317]:     <system>
Apr 14 04:16:31 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:16:31 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:16:31 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:16:31 user nova-compute[71317]:       <entry name="serial">9b532a44-20b7-4590-babe-684c49598a62</entry>
Apr 14 04:16:31 user nova-compute[71317]:       <entry name="uuid">9b532a44-20b7-4590-babe-684c49598a62</entry>
Apr 14 04:16:31 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:16:31 user nova-compute[71317]:     </system>
Apr 14 04:16:31 user nova-compute[71317]:   </sysinfo>
Apr 14 04:16:31 user nova-compute[71317]:   <os>
Apr 14 04:16:31 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:16:31 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:16:31 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:16:31 user nova-compute[71317]:   </os>
Apr 14 04:16:31 user nova-compute[71317]:   <features>
Apr 14 04:16:31 user nova-compute[71317]:     <acpi/>
Apr 14 04:16:31 user nova-compute[71317]:     <apic/>
Apr 14 04:16:31 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:16:31 user nova-compute[71317]:   </features>
Apr 14 04:16:31 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:16:31 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:16:31 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:16:31 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:16:31 user nova-compute[71317]:   </clock>
Apr 14 04:16:31 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:16:31 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:16:31 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:16:31 user nova-compute[71317]:   </cpu>
Apr 14 04:16:31 user nova-compute[71317]:   <devices>
Apr 14 04:16:31 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:16:31 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:16:31 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk"/>
Apr 14 04:16:31 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:16:31 user nova-compute[71317]:     </disk>
Apr 14 04:16:31 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:16:31 user nova-compute[71317]:       <mac address="fa:16:3e:47:9f:40"/>
Apr 14 04:16:31 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:31 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:16:31 user nova-compute[71317]:       <target dev="tapc45f9c53-8d"/>
Apr 14 04:16:31 user nova-compute[71317]:     </interface>
Apr 14 04:16:31 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:16:31 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/console.log" append="off"/>
Apr 14 04:16:31 user nova-compute[71317]:     </serial>
Apr 14 04:16:31 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:16:31 user nova-compute[71317]:     <video>
Apr 14 04:16:31 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:31 user nova-compute[71317]:     </video>
Apr 14 04:16:31 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:16:31 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:16:31 user nova-compute[71317]:     </rng>
Apr 14 04:16:31 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:16:31 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:16:31 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:16:31 user nova-compute[71317]:     </memballoon>
Apr 14 04:16:31 user nova-compute[71317]:   </devices>
Apr 14 04:16:31 user nova-compute[71317]: </domain>
Apr 14 04:16:31 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1687697238',display_name='tempest-ServersNegativeTestJSON-server-1687697238',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1687697238',id=2,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bae0fb4af555422a847620627dffeae8',ramdisk_id='',reservation_id='r-70d12ce1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-406789078',owner_user_name='tempest-ServersNegativeTestJSON-406789078-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:26Z,user_data=None,user_id='ea68fb139b004f748cf1bcfb8d675a2e',uuid=9b532a44-20b7-4590-babe-684c49598a62,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "address": "fa:16:3e:47:9f:40", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc45f9c53-8d", "ovs_interfaceid": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Converting VIF {"id": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "address": "fa:16:3e:47:9f:40", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc45f9c53-8d", "ovs_interfaceid": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:47:9f:40,bridge_name='br-int',has_traffic_filtering=True,id=c45f9c53-8d5c-4b47-8e0a-ebb292cefb06,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc45f9c53-8d') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG os_vif [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:47:9f:40,bridge_name='br-int',has_traffic_filtering=True,id=c45f9c53-8d5c-4b47-8e0a-ebb292cefb06,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc45f9c53-8d') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Created schema index Interface.name {{(pid=71317) autocreate_indices /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Created schema index Port.name {{(pid=71317) autocreate_indices /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 14 04:16:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Created schema index Bridge.name {{(pid=71317) autocreate_indices /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] tcp:127.0.0.1:6640: entering CONNECTING {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [POLLOUT] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:16:32 user nova-compute[71317]: INFO oslo.privsep.daemon [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'vif_plug_ovs.privsep.vif_plug', '--privsep_sock_path', '/tmp/tmpfwv6kpjv/privsep.sock']
Apr 14 04:16:32 user sudo[80077]:    stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpfwv6kpjv/privsep.sock
Apr 14 04:16:32 user sudo[80077]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.network.neutron [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Updating instance_info_cache with network_info: [{"id": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "address": "fa:16:3e:41:ed:a5", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22212b8b-66", "ovs_interfaceid": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Releasing lock "refresh_cache-0ff97860-2441-437c-b118-6089c3cc4160" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Instance network_info: |[{"id": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "address": "fa:16:3e:41:ed:a5", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22212b8b-66", "ovs_interfaceid": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-417129e4-3423-4518-a8f7-a56cbac38a48 req-72fa39a9-c49b-489f-bda4-40b14b1491fd service nova] Acquired lock "refresh_cache-0ff97860-2441-437c-b118-6089c3cc4160" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.network.neutron [req-417129e4-3423-4518-a8f7-a56cbac38a48 req-72fa39a9-c49b-489f-bda4-40b14b1491fd service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Refreshing network info cache for port 22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Start _get_guest_xml network_info=[{"id": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "address": "fa:16:3e:41:ed:a5", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22212b8b-66", "ovs_interfaceid": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:16:32 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:32 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-844517641',display_name='tempest-VolumesActionsTest-instance-844517641',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-844517641',id=1,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='280aa55745a749b7baa134334d25c9c3',ramdisk_id='',reservation_id='r-bl93w3ra',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-447148503',owner_user_name='tempest-VolumesActionsTest-447148503-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:23Z,user_data=None,user_id='6e422970cd8340bf996a135088094f51',uuid=0ff97860-2441-437c-b118-6089c3cc4160,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "address": "fa:16:3e:41:ed:a5", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22212b8b-66", "ovs_interfaceid": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Converting VIF {"id": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "address": "fa:16:3e:41:ed:a5", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22212b8b-66", "ovs_interfaceid": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:41:ed:a5,bridge_name='br-int',has_traffic_filtering=True,id=22212b8b-6656-4d88-ad5c-e0d8f657062f,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22212b8b-66') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.objects.instance [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lazy-loading 'pci_devices' on Instance uuid 0ff97860-2441-437c-b118-6089c3cc4160 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:16:32 user nova-compute[71317]:   <uuid>0ff97860-2441-437c-b118-6089c3cc4160</uuid>
Apr 14 04:16:32 user nova-compute[71317]:   <name>instance-00000001</name>
Apr 14 04:16:32 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:16:32 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:16:32 user nova-compute[71317]:   <metadata>
Apr 14 04:16:32 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:16:32 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:16:32 user nova-compute[71317]:       <nova:name>tempest-VolumesActionsTest-instance-844517641</nova:name>
Apr 14 04:16:32 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:16:32</nova:creationTime>
Apr 14 04:16:32 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:16:32 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:16:32 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:16:32 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:16:32 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:16:32 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:16:32 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:16:32 user nova-compute[71317]:       <nova:owner>
Apr 14 04:16:32 user nova-compute[71317]:         <nova:user uuid="6e422970cd8340bf996a135088094f51">tempest-VolumesActionsTest-447148503-project-member</nova:user>
Apr 14 04:16:32 user nova-compute[71317]:         <nova:project uuid="280aa55745a749b7baa134334d25c9c3">tempest-VolumesActionsTest-447148503</nova:project>
Apr 14 04:16:32 user nova-compute[71317]:       </nova:owner>
Apr 14 04:16:32 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:16:32 user nova-compute[71317]:       <nova:ports>
Apr 14 04:16:32 user nova-compute[71317]:         <nova:port uuid="22212b8b-6656-4d88-ad5c-e0d8f657062f">
Apr 14 04:16:32 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Apr 14 04:16:32 user nova-compute[71317]:         </nova:port>
Apr 14 04:16:32 user nova-compute[71317]:       </nova:ports>
Apr 14 04:16:32 user nova-compute[71317]:     </nova:instance>
Apr 14 04:16:32 user nova-compute[71317]:   </metadata>
Apr 14 04:16:32 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:16:32 user nova-compute[71317]:     <system>
Apr 14 04:16:32 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:16:32 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:16:32 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:16:32 user nova-compute[71317]:       <entry name="serial">0ff97860-2441-437c-b118-6089c3cc4160</entry>
Apr 14 04:16:32 user nova-compute[71317]:       <entry name="uuid">0ff97860-2441-437c-b118-6089c3cc4160</entry>
Apr 14 04:16:32 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:16:32 user nova-compute[71317]:     </system>
Apr 14 04:16:32 user nova-compute[71317]:   </sysinfo>
Apr 14 04:16:32 user nova-compute[71317]:   <os>
Apr 14 04:16:32 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:16:32 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:16:32 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:16:32 user nova-compute[71317]:   </os>
Apr 14 04:16:32 user nova-compute[71317]:   <features>
Apr 14 04:16:32 user nova-compute[71317]:     <acpi/>
Apr 14 04:16:32 user nova-compute[71317]:     <apic/>
Apr 14 04:16:32 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:16:32 user nova-compute[71317]:   </features>
Apr 14 04:16:32 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:16:32 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:16:32 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:16:32 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:16:32 user nova-compute[71317]:   </clock>
Apr 14 04:16:32 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:16:32 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:16:32 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:16:32 user nova-compute[71317]:   </cpu>
Apr 14 04:16:32 user nova-compute[71317]:   <devices>
Apr 14 04:16:32 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:16:32 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:16:32 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk"/>
Apr 14 04:16:32 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:16:32 user nova-compute[71317]:     </disk>
Apr 14 04:16:32 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:16:32 user nova-compute[71317]:       <mac address="fa:16:3e:41:ed:a5"/>
Apr 14 04:16:32 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:32 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:16:32 user nova-compute[71317]:       <target dev="tap22212b8b-66"/>
Apr 14 04:16:32 user nova-compute[71317]:     </interface>
Apr 14 04:16:32 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:16:32 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/console.log" append="off"/>
Apr 14 04:16:32 user nova-compute[71317]:     </serial>
Apr 14 04:16:32 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:16:32 user nova-compute[71317]:     <video>
Apr 14 04:16:32 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:32 user nova-compute[71317]:     </video>
Apr 14 04:16:32 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:16:32 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:16:32 user nova-compute[71317]:     </rng>
Apr 14 04:16:32 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:16:32 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:16:32 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:16:32 user nova-compute[71317]:     </memballoon>
Apr 14 04:16:32 user nova-compute[71317]:   </devices>
Apr 14 04:16:32 user nova-compute[71317]: </domain>
Apr 14 04:16:32 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-844517641',display_name='tempest-VolumesActionsTest-instance-844517641',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-844517641',id=1,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='280aa55745a749b7baa134334d25c9c3',ramdisk_id='',reservation_id='r-bl93w3ra',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-447148503',owner_user_name='tempest-VolumesActionsTest-447148503-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:23Z,user_data=None,user_id='6e422970cd8340bf996a135088094f51',uuid=0ff97860-2441-437c-b118-6089c3cc4160,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "address": "fa:16:3e:41:ed:a5", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22212b8b-66", "ovs_interfaceid": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Converting VIF {"id": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "address": "fa:16:3e:41:ed:a5", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22212b8b-66", "ovs_interfaceid": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:41:ed:a5,bridge_name='br-int',has_traffic_filtering=True,id=22212b8b-6656-4d88-ad5c-e0d8f657062f,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22212b8b-66') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG os_vif [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:41:ed:a5,bridge_name='br-int',has_traffic_filtering=True,id=22212b8b-6656-4d88-ad5c-e0d8f657062f,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22212b8b-66') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:16:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:33 user nova-compute[71317]: DEBUG nova.compute.manager [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:16:34 user sudo[80077]: pam_unix(sudo:session): session closed for user root
Apr 14 04:16:34 user nova-compute[71317]: INFO oslo.privsep.daemon [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Spawned new privsep daemon via rootwrap
Apr 14 04:16:34 user nova-compute[71317]: INFO oslo.privsep.daemon [-] privsep daemon starting
Apr 14 04:16:34 user nova-compute[71317]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Apr 14 04:16:34 user nova-compute[71317]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_DAC_OVERRIDE|CAP_NET_ADMIN/CAP_DAC_OVERRIDE|CAP_NET_ADMIN/none
Apr 14 04:16:34 user nova-compute[71317]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 80091
Apr 14 04:16:34 user nova-compute[71317]: WARNING oslo_privsep.priv_context [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] privsep daemon already running
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:16:34 user nova-compute[71317]: INFO nova.compute.claims [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Claim successful on node user
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.network.neutron [req-417129e4-3423-4518-a8f7-a56cbac38a48 req-72fa39a9-c49b-489f-bda4-40b14b1491fd service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Updated VIF entry in instance network info cache for port 22212b8b-6656-4d88-ad5c-e0d8f657062f. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.network.neutron [req-417129e4-3423-4518-a8f7-a56cbac38a48 req-72fa39a9-c49b-489f-bda4-40b14b1491fd service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Updating instance_info_cache with network_info: [{"id": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "address": "fa:16:3e:41:ed:a5", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22212b8b-66", "ovs_interfaceid": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-417129e4-3423-4518-a8f7-a56cbac38a48 req-72fa39a9-c49b-489f-bda4-40b14b1491fd service nova] Releasing lock "refresh_cache-0ff97860-2441-437c-b118-6089c3cc4160" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.network.neutron [req-ac4ff65a-557a-4c8d-a64e-96e319749867 req-45263051-7492-400d-9f96-5b461fdcb6d7 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Updated VIF entry in instance network info cache for port c45f9c53-8d5c-4b47-8e0a-ebb292cefb06. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.network.neutron [req-ac4ff65a-557a-4c8d-a64e-96e319749867 req-45263051-7492-400d-9f96-5b461fdcb6d7 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Updating instance_info_cache with network_info: [{"id": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "address": "fa:16:3e:47:9f:40", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc45f9c53-8d", "ovs_interfaceid": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ac4ff65a-557a-4c8d-a64e-96e319749867 req-45263051-7492-400d-9f96-5b461fdcb6d7 service nova] Releasing lock "refresh_cache-9b532a44-20b7-4590-babe-684c49598a62" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.340s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.network.neutron [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:16:34 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap22212b8b-66, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap22212b8b-66, col_values=(('external_ids', {'iface-id': '22212b8b-6656-4d88-ad5c-e0d8f657062f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:41:ed:a5', 'vm-uuid': '0ff97860-2441-437c-b118-6089c3cc4160'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:16:34 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Creating image(s)
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "/opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "/opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "/opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:34 user nova-compute[71317]: INFO os_vif [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:41:ed:a5,bridge_name='br-int',has_traffic_filtering=True,id=22212b8b-6656-4d88-ad5c-e0d8f657062f,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22212b8b-66')
Apr 14 04:16:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc45f9c53-8d, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc45f9c53-8d, col_values=(('external_ids', {'iface-id': 'c45f9c53-8d5c-4b47-8e0a-ebb292cefb06', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:47:9f:40', 'vm-uuid': '9b532a44-20b7-4590-babe-684c49598a62'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:16:34 user nova-compute[71317]: INFO os_vif [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:47:9f:40,bridge_name='br-int',has_traffic_filtering=True,id=c45f9c53-8d5c-4b47-8e0a-ebb292cefb06,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc45f9c53-8d')
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] No VIF found with MAC fa:16:3e:41:ed:a5, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] No VIF found with MAC fa:16:3e:47:9f:40, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG nova.policy [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7610b45003e34f3395268977612f6c94', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a89255339f74e55ad7c92967558bb81', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.129s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk 1073741824" returned: 0 in 0.049s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.182s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Checking if we can resize image /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Cannot resize image /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG nova.objects.instance [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'migration_context' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Ensure instance console log exists: /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:37 user nova-compute[71317]: DEBUG nova.network.neutron [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Successfully created port: df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:16:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "e37e27b4-0813-4df9-ac66-1516334c12cb" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-51414bde-9998-4cd9-98c4-270144332180 req-ffc7f5ab-289f-4f5e-a3db-b0ee2a27a4ff service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Received event network-vif-plugged-22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-51414bde-9998-4cd9-98c4-270144332180 req-ffc7f5ab-289f-4f5e-a3db-b0ee2a27a4ff service nova] Acquiring lock "0ff97860-2441-437c-b118-6089c3cc4160-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-51414bde-9998-4cd9-98c4-270144332180 req-ffc7f5ab-289f-4f5e-a3db-b0ee2a27a4ff service nova] Lock "0ff97860-2441-437c-b118-6089c3cc4160-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-51414bde-9998-4cd9-98c4-270144332180 req-ffc7f5ab-289f-4f5e-a3db-b0ee2a27a4ff service nova] Lock "0ff97860-2441-437c-b118-6089c3cc4160-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-51414bde-9998-4cd9-98c4-270144332180 req-ffc7f5ab-289f-4f5e-a3db-b0ee2a27a4ff service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] No waiting events found dispatching network-vif-plugged-22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:38 user nova-compute[71317]: WARNING nova.compute.manager [req-51414bde-9998-4cd9-98c4-270144332180 req-ffc7f5ab-289f-4f5e-a3db-b0ee2a27a4ff service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Received unexpected event network-vif-plugged-22212b8b-6656-4d88-ad5c-e0d8f657062f for instance with vm_state building and task_state spawning.
Apr 14 04:16:38 user nova-compute[71317]: DEBUG nova.compute.manager [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:16:38 user nova-compute[71317]: INFO nova.compute.claims [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Claim successful on node user
Apr 14 04:16:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.358s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG nova.compute.manager [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG nova.compute.manager [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG nova.network.neutron [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:16:38 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:16:38 user nova-compute[71317]: DEBUG nova.compute.manager [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG nova.compute.manager [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:16:38 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Creating image(s)
Apr 14 04:16:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "/opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "/opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "/opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.160s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.policy [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'fab852bb302f44b1af63083c99ea244c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce5e634f187b470085e7bfb3c1cfcbe7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.164s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445799.3113651, 9b532a44-20b7-4590-babe-684c49598a62 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:16:39 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9b532a44-20b7-4590-babe-684c49598a62] VM Resumed (Lifecycle Event)
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:16:39 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Instance spawned successfully.
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:16:39 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Instance spawned successfully.
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk 1073741824" returned: 0 in 0.078s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.246s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.178s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Checking if we can resize image /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:39 user nova-compute[71317]: INFO nova.compute.manager [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Took 16.20 seconds to spawn the instance on the hypervisor.
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:16:39 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9b532a44-20b7-4590-babe-684c49598a62] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445799.3126156, 9b532a44-20b7-4590-babe-684c49598a62 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:16:39 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9b532a44-20b7-4590-babe-684c49598a62] VM Started (Lifecycle Event)
Apr 14 04:16:39 user nova-compute[71317]: INFO nova.compute.manager [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Took 13.49 seconds to spawn the instance on the hypervisor.
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:16:39 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9b532a44-20b7-4590-babe-684c49598a62] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445799.3145506, 0ff97860-2441-437c-b118-6089c3cc4160 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:16:39 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] VM Resumed (Lifecycle Event)
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Cannot resize image /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.objects.instance [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lazy-loading 'migration_context' on Instance uuid e37e27b4-0813-4df9-ac66-1516334c12cb {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Ensure instance console log exists: /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:39 user nova-compute[71317]: INFO nova.compute.manager [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Took 14.24 seconds to build instance.
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-08a8d46c-70b4-4f37-884c-7809afae0e93 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "9b532a44-20b7-4590-babe-684c49598a62" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 14.376s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:39 user nova-compute[71317]: INFO nova.compute.manager [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Took 17.25 seconds to build instance.
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445799.3367712, 0ff97860-2441-437c-b118-6089c3cc4160 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:16:39 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] VM Started (Lifecycle Event)
Apr 14 04:16:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-30edb0e7-f91e-4b6f-9711-7074ee219958 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "0ff97860-2441-437c-b118-6089c3cc4160" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 17.500s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:16:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:16:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-10e33f75-10de-4436-84db-6d5b7f7b3b56 req-6742e0b8-01da-414b-ba94-b7526a330f05 service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Received event network-vif-plugged-22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-10e33f75-10de-4436-84db-6d5b7f7b3b56 req-6742e0b8-01da-414b-ba94-b7526a330f05 service nova] Acquiring lock "0ff97860-2441-437c-b118-6089c3cc4160-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-10e33f75-10de-4436-84db-6d5b7f7b3b56 req-6742e0b8-01da-414b-ba94-b7526a330f05 service nova] Lock "0ff97860-2441-437c-b118-6089c3cc4160-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-10e33f75-10de-4436-84db-6d5b7f7b3b56 req-6742e0b8-01da-414b-ba94-b7526a330f05 service nova] Lock "0ff97860-2441-437c-b118-6089c3cc4160-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-10e33f75-10de-4436-84db-6d5b7f7b3b56 req-6742e0b8-01da-414b-ba94-b7526a330f05 service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] No waiting events found dispatching network-vif-plugged-22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:40 user nova-compute[71317]: WARNING nova.compute.manager [req-10e33f75-10de-4436-84db-6d5b7f7b3b56 req-6742e0b8-01da-414b-ba94-b7526a330f05 service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Received unexpected event network-vif-plugged-22212b8b-6656-4d88-ad5c-e0d8f657062f for instance with vm_state active and task_state None.
Apr 14 04:16:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-ab3df53d-3a74-4d81-b4e8-d2ad535ec82e req-32484043-0f72-4e7f-be3a-3775b632652f service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Received event network-vif-plugged-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ab3df53d-3a74-4d81-b4e8-d2ad535ec82e req-32484043-0f72-4e7f-be3a-3775b632652f service nova] Acquiring lock "9b532a44-20b7-4590-babe-684c49598a62-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ab3df53d-3a74-4d81-b4e8-d2ad535ec82e req-32484043-0f72-4e7f-be3a-3775b632652f service nova] Lock "9b532a44-20b7-4590-babe-684c49598a62-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ab3df53d-3a74-4d81-b4e8-d2ad535ec82e req-32484043-0f72-4e7f-be3a-3775b632652f service nova] Lock "9b532a44-20b7-4590-babe-684c49598a62-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-ab3df53d-3a74-4d81-b4e8-d2ad535ec82e req-32484043-0f72-4e7f-be3a-3775b632652f service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] No waiting events found dispatching network-vif-plugged-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:40 user nova-compute[71317]: WARNING nova.compute.manager [req-ab3df53d-3a74-4d81-b4e8-d2ad535ec82e req-32484043-0f72-4e7f-be3a-3775b632652f service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Received unexpected event network-vif-plugged-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 for instance with vm_state active and task_state None.
Apr 14 04:16:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.network.neutron [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Successfully updated port: df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.004s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquired lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.network.neutron [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:16:42 user nova-compute[71317]: INFO nova.compute.claims [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Claim successful on node user
Apr 14 04:16:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.network.neutron [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Successfully created port: 2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.network.neutron [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Received event network-vif-plugged-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] Acquiring lock "9b532a44-20b7-4590-babe-684c49598a62-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] Lock "9b532a44-20b7-4590-babe-684c49598a62-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] Lock "9b532a44-20b7-4590-babe-684c49598a62-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] No waiting events found dispatching network-vif-plugged-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:42 user nova-compute[71317]: WARNING nova.compute.manager [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Received unexpected event network-vif-plugged-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 for instance with vm_state active and task_state None.
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-changed-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Refreshing instance network info cache due to event network-changed-df63af46-27d1-4086-ae2c-39e8bc829bc7. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] Acquiring lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:16:42 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.591s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.155s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:16:43 user nova-compute[71317]: INFO nova.compute.claims [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Claim successful on node user
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.network.neutron [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:16:43 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:16:43 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Creating image(s)
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "/opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "/opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "/opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.575s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.240s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.policy [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '5234914aec8e443b8a38ac6b38729031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33320dc2f2c14ecfa5abf6574029c36c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.network.neutron [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:16:43 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:16:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.191s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk 1073741824" returned: 0 in 0.079s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.274s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.compute.manager [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:16:44 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Creating image(s)
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.228s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Checking if we can resize image /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.201s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.policy [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e35b3ae371cf4dd4b21e2999e58cd2f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c14c332dfcc44b49ddf3cb482dd0d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.184s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.network.neutron [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updating instance_info_cache with network_info: [{"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.332s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Cannot resize image /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'migration_context' on Instance uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk 1073741824" returned: 0 in 0.094s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Ensure instance console log exists: /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.290s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Releasing lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.compute.manager [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance network_info: |[{"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] Acquired lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.network.neutron [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Refreshing network info cache for port df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Start _get_guest_xml network_info=[{"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:16:44 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:44 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1781752187',display_name='tempest-AttachVolumeTestJSON-server-1781752187',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1781752187',id=3,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBY2UT8M9EXLRm2waCk4jzyD2YisXrBUU8nZo6LT6g0AlR0LbKytQJ1iJL1X5k+1hK+8YQxwsn1kgdkATbgdoU6lyDlwBLIIaqdqxbWOahpeOBX2u8LLHQwm2K/x2XVeAg==',key_name='tempest-keypair-1428467741',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6a89255339f74e55ad7c92967558bb81',ramdisk_id='',reservation_id='r-snsinryu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-991906288',owner_user_name='tempest-AttachVolumeTestJSON-991906288-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:35Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7610b45003e34f3395268977612f6c94',uuid=da3b4651-aaa0-430d-85b8-c15de386d08f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converting VIF {"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.objects.instance [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'pci_devices' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.218s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Checking if we can resize image /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:16:44 user nova-compute[71317]:   <uuid>da3b4651-aaa0-430d-85b8-c15de386d08f</uuid>
Apr 14 04:16:44 user nova-compute[71317]:   <name>instance-00000003</name>
Apr 14 04:16:44 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:16:44 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:16:44 user nova-compute[71317]:   <metadata>
Apr 14 04:16:44 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:16:44 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:16:44 user nova-compute[71317]:       <nova:name>tempest-AttachVolumeTestJSON-server-1781752187</nova:name>
Apr 14 04:16:44 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:16:44</nova:creationTime>
Apr 14 04:16:44 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:16:44 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:16:44 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:16:44 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:16:44 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:16:44 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:16:44 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:16:44 user nova-compute[71317]:       <nova:owner>
Apr 14 04:16:44 user nova-compute[71317]:         <nova:user uuid="7610b45003e34f3395268977612f6c94">tempest-AttachVolumeTestJSON-991906288-project-member</nova:user>
Apr 14 04:16:44 user nova-compute[71317]:         <nova:project uuid="6a89255339f74e55ad7c92967558bb81">tempest-AttachVolumeTestJSON-991906288</nova:project>
Apr 14 04:16:44 user nova-compute[71317]:       </nova:owner>
Apr 14 04:16:44 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:16:44 user nova-compute[71317]:       <nova:ports>
Apr 14 04:16:44 user nova-compute[71317]:         <nova:port uuid="df63af46-27d1-4086-ae2c-39e8bc829bc7">
Apr 14 04:16:44 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 04:16:44 user nova-compute[71317]:         </nova:port>
Apr 14 04:16:44 user nova-compute[71317]:       </nova:ports>
Apr 14 04:16:44 user nova-compute[71317]:     </nova:instance>
Apr 14 04:16:44 user nova-compute[71317]:   </metadata>
Apr 14 04:16:44 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:16:44 user nova-compute[71317]:     <system>
Apr 14 04:16:44 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:16:44 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:16:44 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:16:44 user nova-compute[71317]:       <entry name="serial">da3b4651-aaa0-430d-85b8-c15de386d08f</entry>
Apr 14 04:16:44 user nova-compute[71317]:       <entry name="uuid">da3b4651-aaa0-430d-85b8-c15de386d08f</entry>
Apr 14 04:16:44 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:16:44 user nova-compute[71317]:     </system>
Apr 14 04:16:44 user nova-compute[71317]:   </sysinfo>
Apr 14 04:16:44 user nova-compute[71317]:   <os>
Apr 14 04:16:44 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:16:44 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:16:44 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:16:44 user nova-compute[71317]:   </os>
Apr 14 04:16:44 user nova-compute[71317]:   <features>
Apr 14 04:16:44 user nova-compute[71317]:     <acpi/>
Apr 14 04:16:44 user nova-compute[71317]:     <apic/>
Apr 14 04:16:44 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:16:44 user nova-compute[71317]:   </features>
Apr 14 04:16:44 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:16:44 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:16:44 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:16:44 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:16:44 user nova-compute[71317]:   </clock>
Apr 14 04:16:44 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:16:44 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:16:44 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:16:44 user nova-compute[71317]:   </cpu>
Apr 14 04:16:44 user nova-compute[71317]:   <devices>
Apr 14 04:16:44 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:16:44 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:16:44 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk"/>
Apr 14 04:16:44 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:16:44 user nova-compute[71317]:     </disk>
Apr 14 04:16:44 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:16:44 user nova-compute[71317]:       <mac address="fa:16:3e:68:ca:2f"/>
Apr 14 04:16:44 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:44 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:16:44 user nova-compute[71317]:       <target dev="tapdf63af46-27"/>
Apr 14 04:16:44 user nova-compute[71317]:     </interface>
Apr 14 04:16:44 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:16:44 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/console.log" append="off"/>
Apr 14 04:16:44 user nova-compute[71317]:     </serial>
Apr 14 04:16:44 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:16:44 user nova-compute[71317]:     <video>
Apr 14 04:16:44 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:44 user nova-compute[71317]:     </video>
Apr 14 04:16:44 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:16:44 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:16:44 user nova-compute[71317]:     </rng>
Apr 14 04:16:44 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:16:44 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:16:44 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:16:44 user nova-compute[71317]:     </memballoon>
Apr 14 04:16:44 user nova-compute[71317]:   </devices>
Apr 14 04:16:44 user nova-compute[71317]: </domain>
Apr 14 04:16:44 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1781752187',display_name='tempest-AttachVolumeTestJSON-server-1781752187',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1781752187',id=3,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBY2UT8M9EXLRm2waCk4jzyD2YisXrBUU8nZo6LT6g0AlR0LbKytQJ1iJL1X5k+1hK+8YQxwsn1kgdkATbgdoU6lyDlwBLIIaqdqxbWOahpeOBX2u8LLHQwm2K/x2XVeAg==',key_name='tempest-keypair-1428467741',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6a89255339f74e55ad7c92967558bb81',ramdisk_id='',reservation_id='r-snsinryu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-991906288',owner_user_name='tempest-AttachVolumeTestJSON-991906288-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:35Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7610b45003e34f3395268977612f6c94',uuid=da3b4651-aaa0-430d-85b8-c15de386d08f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converting VIF {"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG os_vif [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapdf63af46-27, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapdf63af46-27, col_values=(('external_ids', {'iface-id': 'df63af46-27d1-4086-ae2c-39e8bc829bc7', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:68:ca:2f', 'vm-uuid': 'da3b4651-aaa0-430d-85b8-c15de386d08f'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:16:44 user nova-compute[71317]: INFO os_vif [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27')
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Cannot resize image /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.objects.instance [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'migration_context' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Ensure instance console log exists: /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:16:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No VIF found with MAC fa:16:3e:68:ca:2f, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:16:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:16:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:45 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:16:45 user nova-compute[71317]: INFO nova.compute.claims [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Claim successful on node user
Apr 14 04:16:45 user nova-compute[71317]: DEBUG nova.network.neutron [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Successfully updated port: 2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:16:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "refresh_cache-e37e27b4-0813-4df9-ac66-1516334c12cb" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquired lock "refresh_cache-e37e27b4-0813-4df9-ac66-1516334c12cb" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:45 user nova-compute[71317]: DEBUG nova.network.neutron [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-395aef29-4a39-4656-8ea2-db2a81ac9959 req-be1cf71e-ff66-4c9d-b5c8-e33774e0c5c0 service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Received event network-changed-2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-395aef29-4a39-4656-8ea2-db2a81ac9959 req-be1cf71e-ff66-4c9d-b5c8-e33774e0c5c0 service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Refreshing instance network info cache due to event network-changed-2a342575-05e3-4179-a370-6d39ac4f815d. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-395aef29-4a39-4656-8ea2-db2a81ac9959 req-be1cf71e-ff66-4c9d-b5c8-e33774e0c5c0 service nova] Acquiring lock "refresh_cache-e37e27b4-0813-4df9-ac66-1516334c12cb" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.710s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.network.neutron [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.network.neutron [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updated VIF entry in instance network info cache for port df63af46-27d1-4086-ae2c-39e8bc829bc7. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.network.neutron [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updating instance_info_cache with network_info: [{"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-495df739-5682-4926-93e2-99dc8cf9979f req-2c3b2454-238d-4bfc-b032-e5b2808cbf24 service nova] Releasing lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.network.neutron [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:16:46 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.policy [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '5234914aec8e443b8a38ac6b38729031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33320dc2f2c14ecfa5abf6574029c36c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:16:46 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Creating image(s)
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "/opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "/opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "/opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG nova.network.neutron [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Successfully created port: 593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.182s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.159s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk 1073741824" returned: 0 in 0.082s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.251s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.188s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Checking if we can resize image /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.243s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Cannot resize image /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.objects.instance [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'migration_context' on Instance uuid 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Ensure instance console log exists: /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Updating instance_info_cache with network_info: [{"id": "2a342575-05e3-4179-a370-6d39ac4f815d", "address": "fa:16:3e:61:5e:bb", "network": {"id": "2067a33c-66ab-4b9b-a1d5-a8dbb603f119", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1353356979-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ce5e634f187b470085e7bfb3c1cfcbe7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2a342575-05", "ovs_interfaceid": "2a342575-05e3-4179-a370-6d39ac4f815d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Releasing lock "refresh_cache-e37e27b4-0813-4df9-ac66-1516334c12cb" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Instance network_info: |[{"id": "2a342575-05e3-4179-a370-6d39ac4f815d", "address": "fa:16:3e:61:5e:bb", "network": {"id": "2067a33c-66ab-4b9b-a1d5-a8dbb603f119", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1353356979-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ce5e634f187b470085e7bfb3c1cfcbe7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2a342575-05", "ovs_interfaceid": "2a342575-05e3-4179-a370-6d39ac4f815d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-395aef29-4a39-4656-8ea2-db2a81ac9959 req-be1cf71e-ff66-4c9d-b5c8-e33774e0c5c0 service nova] Acquired lock "refresh_cache-e37e27b4-0813-4df9-ac66-1516334c12cb" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.network.neutron [req-395aef29-4a39-4656-8ea2-db2a81ac9959 req-be1cf71e-ff66-4c9d-b5c8-e33774e0c5c0 service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Refreshing network info cache for port 2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Start _get_guest_xml network_info=[{"id": "2a342575-05e3-4179-a370-6d39ac4f815d", "address": "fa:16:3e:61:5e:bb", "network": {"id": "2067a33c-66ab-4b9b-a1d5-a8dbb603f119", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1353356979-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ce5e634f187b470085e7bfb3c1cfcbe7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2a342575-05", "ovs_interfaceid": "2a342575-05e3-4179-a370-6d39ac4f815d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:16:47 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:47 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-238847393',display_name='tempest-DeleteServersTestJSON-server-238847393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-238847393',id=4,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ce5e634f187b470085e7bfb3c1cfcbe7',ramdisk_id='',reservation_id='r-2fxy9zb2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-DeleteServersTestJSON-2046031715',owner_user_name='tempest-DeleteServersTestJSON-2046031715-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:39Z,user_data=None,user_id='fab852bb302f44b1af63083c99ea244c',uuid=e37e27b4-0813-4df9-ac66-1516334c12cb,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2a342575-05e3-4179-a370-6d39ac4f815d", "address": "fa:16:3e:61:5e:bb", "network": {"id": "2067a33c-66ab-4b9b-a1d5-a8dbb603f119", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1353356979-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ce5e634f187b470085e7bfb3c1cfcbe7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2a342575-05", "ovs_interfaceid": "2a342575-05e3-4179-a370-6d39ac4f815d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Converting VIF {"id": "2a342575-05e3-4179-a370-6d39ac4f815d", "address": "fa:16:3e:61:5e:bb", "network": {"id": "2067a33c-66ab-4b9b-a1d5-a8dbb603f119", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1353356979-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ce5e634f187b470085e7bfb3c1cfcbe7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2a342575-05", "ovs_interfaceid": "2a342575-05e3-4179-a370-6d39ac4f815d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:61:5e:bb,bridge_name='br-int',has_traffic_filtering=True,id=2a342575-05e3-4179-a370-6d39ac4f815d,network=Network(2067a33c-66ab-4b9b-a1d5-a8dbb603f119),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2a342575-05') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.objects.instance [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lazy-loading 'pci_devices' on Instance uuid e37e27b4-0813-4df9-ac66-1516334c12cb {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:16:47 user nova-compute[71317]:   <uuid>e37e27b4-0813-4df9-ac66-1516334c12cb</uuid>
Apr 14 04:16:47 user nova-compute[71317]:   <name>instance-00000004</name>
Apr 14 04:16:47 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:16:47 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:16:47 user nova-compute[71317]:   <metadata>
Apr 14 04:16:47 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:16:47 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:16:47 user nova-compute[71317]:       <nova:name>tempest-DeleteServersTestJSON-server-238847393</nova:name>
Apr 14 04:16:47 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:16:47</nova:creationTime>
Apr 14 04:16:47 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:16:47 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:16:47 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:16:47 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:16:47 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:16:47 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:16:47 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:16:47 user nova-compute[71317]:       <nova:owner>
Apr 14 04:16:47 user nova-compute[71317]:         <nova:user uuid="fab852bb302f44b1af63083c99ea244c">tempest-DeleteServersTestJSON-2046031715-project-member</nova:user>
Apr 14 04:16:47 user nova-compute[71317]:         <nova:project uuid="ce5e634f187b470085e7bfb3c1cfcbe7">tempest-DeleteServersTestJSON-2046031715</nova:project>
Apr 14 04:16:47 user nova-compute[71317]:       </nova:owner>
Apr 14 04:16:47 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:16:47 user nova-compute[71317]:       <nova:ports>
Apr 14 04:16:47 user nova-compute[71317]:         <nova:port uuid="2a342575-05e3-4179-a370-6d39ac4f815d">
Apr 14 04:16:47 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 04:16:47 user nova-compute[71317]:         </nova:port>
Apr 14 04:16:47 user nova-compute[71317]:       </nova:ports>
Apr 14 04:16:47 user nova-compute[71317]:     </nova:instance>
Apr 14 04:16:47 user nova-compute[71317]:   </metadata>
Apr 14 04:16:47 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:16:47 user nova-compute[71317]:     <system>
Apr 14 04:16:47 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:16:47 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:16:47 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:16:47 user nova-compute[71317]:       <entry name="serial">e37e27b4-0813-4df9-ac66-1516334c12cb</entry>
Apr 14 04:16:47 user nova-compute[71317]:       <entry name="uuid">e37e27b4-0813-4df9-ac66-1516334c12cb</entry>
Apr 14 04:16:47 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:16:47 user nova-compute[71317]:     </system>
Apr 14 04:16:47 user nova-compute[71317]:   </sysinfo>
Apr 14 04:16:47 user nova-compute[71317]:   <os>
Apr 14 04:16:47 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:16:47 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:16:47 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:16:47 user nova-compute[71317]:   </os>
Apr 14 04:16:47 user nova-compute[71317]:   <features>
Apr 14 04:16:47 user nova-compute[71317]:     <acpi/>
Apr 14 04:16:47 user nova-compute[71317]:     <apic/>
Apr 14 04:16:47 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:16:47 user nova-compute[71317]:   </features>
Apr 14 04:16:47 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:16:47 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:16:47 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:16:47 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:16:47 user nova-compute[71317]:   </clock>
Apr 14 04:16:47 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:16:47 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:16:47 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:16:47 user nova-compute[71317]:   </cpu>
Apr 14 04:16:47 user nova-compute[71317]:   <devices>
Apr 14 04:16:47 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:16:47 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:16:47 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk"/>
Apr 14 04:16:47 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:16:47 user nova-compute[71317]:     </disk>
Apr 14 04:16:47 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:16:47 user nova-compute[71317]:       <mac address="fa:16:3e:61:5e:bb"/>
Apr 14 04:16:47 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:47 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:16:47 user nova-compute[71317]:       <target dev="tap2a342575-05"/>
Apr 14 04:16:47 user nova-compute[71317]:     </interface>
Apr 14 04:16:47 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:16:47 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/console.log" append="off"/>
Apr 14 04:16:47 user nova-compute[71317]:     </serial>
Apr 14 04:16:47 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:16:47 user nova-compute[71317]:     <video>
Apr 14 04:16:47 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:47 user nova-compute[71317]:     </video>
Apr 14 04:16:47 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:16:47 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:16:47 user nova-compute[71317]:     </rng>
Apr 14 04:16:47 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:16:47 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:16:47 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:16:47 user nova-compute[71317]:     </memballoon>
Apr 14 04:16:47 user nova-compute[71317]:   </devices>
Apr 14 04:16:47 user nova-compute[71317]: </domain>
Apr 14 04:16:47 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-238847393',display_name='tempest-DeleteServersTestJSON-server-238847393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-238847393',id=4,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ce5e634f187b470085e7bfb3c1cfcbe7',ramdisk_id='',reservation_id='r-2fxy9zb2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-DeleteServersTestJSON-2046031715',owner_user_name='tempest-DeleteServersTestJSON-2046031715-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:39Z,user_data=None,user_id='fab852bb302f44b1af63083c99ea244c',uuid=e37e27b4-0813-4df9-ac66-1516334c12cb,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2a342575-05e3-4179-a370-6d39ac4f815d", "address": "fa:16:3e:61:5e:bb", "network": {"id": "2067a33c-66ab-4b9b-a1d5-a8dbb603f119", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1353356979-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ce5e634f187b470085e7bfb3c1cfcbe7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2a342575-05", "ovs_interfaceid": "2a342575-05e3-4179-a370-6d39ac4f815d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Converting VIF {"id": "2a342575-05e3-4179-a370-6d39ac4f815d", "address": "fa:16:3e:61:5e:bb", "network": {"id": "2067a33c-66ab-4b9b-a1d5-a8dbb603f119", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1353356979-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ce5e634f187b470085e7bfb3c1cfcbe7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2a342575-05", "ovs_interfaceid": "2a342575-05e3-4179-a370-6d39ac4f815d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:61:5e:bb,bridge_name='br-int',has_traffic_filtering=True,id=2a342575-05e3-4179-a370-6d39ac4f815d,network=Network(2067a33c-66ab-4b9b-a1d5-a8dbb603f119),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2a342575-05') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG os_vif [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:61:5e:bb,bridge_name='br-int',has_traffic_filtering=True,id=2a342575-05e3-4179-a370-6d39ac4f815d,network=Network(2067a33c-66ab-4b9b-a1d5-a8dbb603f119),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2a342575-05') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2a342575-05, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2a342575-05, col_values=(('external_ids', {'iface-id': '2a342575-05e3-4179-a370-6d39ac4f815d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:61:5e:bb', 'vm-uuid': 'e37e27b4-0813-4df9-ac66-1516334c12cb'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:16:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:47 user nova-compute[71317]: INFO os_vif [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:61:5e:bb,bridge_name='br-int',has_traffic_filtering=True,id=2a342575-05e3-4179-a370-6d39ac4f815d,network=Network(2067a33c-66ab-4b9b-a1d5-a8dbb603f119),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2a342575-05')
Apr 14 04:16:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Successfully created port: 3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] No VIF found with MAC fa:16:3e:61:5e:bb, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-e3891da0-1508-480d-9ee3-f28d38e36060 req-dcc325c6-e9ff-4d6c-b01b-749b6cb10736 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e3891da0-1508-480d-9ee3-f28d38e36060 req-dcc325c6-e9ff-4d6c-b01b-749b6cb10736 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e3891da0-1508-480d-9ee3-f28d38e36060 req-dcc325c6-e9ff-4d6c-b01b-749b6cb10736 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e3891da0-1508-480d-9ee3-f28d38e36060 req-dcc325c6-e9ff-4d6c-b01b-749b6cb10736 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-e3891da0-1508-480d-9ee3-f28d38e36060 req-dcc325c6-e9ff-4d6c-b01b-749b6cb10736 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:48 user nova-compute[71317]: WARNING nova.compute.manager [req-e3891da0-1508-480d-9ee3-f28d38e36060 req-dcc325c6-e9ff-4d6c-b01b-749b6cb10736 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state building and task_state spawning.
Apr 14 04:16:48 user nova-compute[71317]: DEBUG nova.network.neutron [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Successfully created port: 1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:16:48 user nova-compute[71317]: INFO nova.compute.claims [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Claim successful on node user
Apr 14 04:16:48 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.507s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445809.1230621, da3b4651-aaa0-430d-85b8-c15de386d08f => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:16:49 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] VM Resumed (Lifecycle Event)
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:16:49 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:16:49 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance spawned successfully.
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:49 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445809.1231925, da3b4651-aaa0-430d-85b8-c15de386d08f => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:16:49 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] VM Started (Lifecycle Event)
Apr 14 04:16:49 user nova-compute[71317]: INFO nova.compute.manager [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Took 14.56 seconds to spawn the instance on the hypervisor.
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:16:49 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Creating image(s)
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "/opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "/opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "/opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:16:49 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:16:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:49 user nova-compute[71317]: INFO nova.compute.manager [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Took 15.42 seconds to build instance.
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.213s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-55e086f4-a6e5-496f-80c4-46501ae83824 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 15.763s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG nova.policy [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '676d80d3684045298d895555247174af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '15bb9ddec81e4f6cbceb6ce7dee9c3a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.200s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk 1073741824" returned: 0 in 0.047s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.253s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Checking if we can resize image /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.197s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Cannot resize image /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.objects.instance [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'migration_context' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Ensure instance console log exists: /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.network.neutron [req-395aef29-4a39-4656-8ea2-db2a81ac9959 req-be1cf71e-ff66-4c9d-b5c8-e33774e0c5c0 service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Updated VIF entry in instance network info cache for port 2a342575-05e3-4179-a370-6d39ac4f815d. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.network.neutron [req-395aef29-4a39-4656-8ea2-db2a81ac9959 req-be1cf71e-ff66-4c9d-b5c8-e33774e0c5c0 service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Updating instance_info_cache with network_info: [{"id": "2a342575-05e3-4179-a370-6d39ac4f815d", "address": "fa:16:3e:61:5e:bb", "network": {"id": "2067a33c-66ab-4b9b-a1d5-a8dbb603f119", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1353356979-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ce5e634f187b470085e7bfb3c1cfcbe7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2a342575-05", "ovs_interfaceid": "2a342575-05e3-4179-a370-6d39ac4f815d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-395aef29-4a39-4656-8ea2-db2a81ac9959 req-be1cf71e-ff66-4c9d-b5c8-e33774e0c5c0 service nova] Releasing lock "refresh_cache-e37e27b4-0813-4df9-ac66-1516334c12cb" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.compute.manager [req-4ec13115-161a-4aa6-9f9c-9a9a90e53f56 req-69e4541b-7f4d-40c9-9cb2-657847625cb9 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4ec13115-161a-4aa6-9f9c-9a9a90e53f56 req-69e4541b-7f4d-40c9-9cb2-657847625cb9 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4ec13115-161a-4aa6-9f9c-9a9a90e53f56 req-69e4541b-7f4d-40c9-9cb2-657847625cb9 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4ec13115-161a-4aa6-9f9c-9a9a90e53f56 req-69e4541b-7f4d-40c9-9cb2-657847625cb9 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.compute.manager [req-4ec13115-161a-4aa6-9f9c-9a9a90e53f56 req-69e4541b-7f4d-40c9-9cb2-657847625cb9 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:50 user nova-compute[71317]: WARNING nova.compute.manager [req-4ec13115-161a-4aa6-9f9c-9a9a90e53f56 req-69e4541b-7f4d-40c9-9cb2-657847625cb9 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state active and task_state None.
Apr 14 04:16:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.network.neutron [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Successfully updated port: 1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquired lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.network.neutron [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.compute.manager [req-ff9b7939-6f4a-44fc-92ea-284574a82dec req-7672bbac-a7a2-45ed-bcb6-695501bb33ad service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Received event network-vif-plugged-2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ff9b7939-6f4a-44fc-92ea-284574a82dec req-7672bbac-a7a2-45ed-bcb6-695501bb33ad service nova] Acquiring lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ff9b7939-6f4a-44fc-92ea-284574a82dec req-7672bbac-a7a2-45ed-bcb6-695501bb33ad service nova] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ff9b7939-6f4a-44fc-92ea-284574a82dec req-7672bbac-a7a2-45ed-bcb6-695501bb33ad service nova] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.compute.manager [req-ff9b7939-6f4a-44fc-92ea-284574a82dec req-7672bbac-a7a2-45ed-bcb6-695501bb33ad service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] No waiting events found dispatching network-vif-plugged-2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:50 user nova-compute[71317]: WARNING nova.compute.manager [req-ff9b7939-6f4a-44fc-92ea-284574a82dec req-7672bbac-a7a2-45ed-bcb6-695501bb33ad service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Received unexpected event network-vif-plugged-2a342575-05e3-4179-a370-6d39ac4f815d for instance with vm_state building and task_state spawning.
Apr 14 04:16:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.network.neutron [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Successfully updated port: 593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquired lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.network.neutron [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.network.neutron [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:16:50 user nova-compute[71317]: DEBUG nova.network.neutron [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.network.neutron [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Successfully updated port: 3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquired lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.network.neutron [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.network.neutron [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Updating instance_info_cache with network_info: [{"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Releasing lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Instance network_info: |[{"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Start _get_guest_xml network_info=[{"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:16:51 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:51 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1602894262',display_name='tempest-ServerRescueNegativeTestJSON-server-1602894262',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1602894262',id=7,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='33320dc2f2c14ecfa5abf6574029c36c',ramdisk_id='',reservation_id='r-5xk8owm0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-508041341',owner_user_name='tempest-ServerRescueNegativeTestJSON-508041341-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:46Z,user_data=None,user_id='5234914aec8e443b8a38ac6b38729031',uuid=2f7e20cd-33ef-4f83-bcdb-a13b9b51370a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converting VIF {"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f5:07:c5,bridge_name='br-int',has_traffic_filtering=True,id=1201ab18-d7ef-4a92-a77e-d0ab69858538,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1201ab18-d7') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.objects.instance [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'pci_devices' on Instance uuid 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.network.neutron [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:16:51 user nova-compute[71317]:   <uuid>2f7e20cd-33ef-4f83-bcdb-a13b9b51370a</uuid>
Apr 14 04:16:51 user nova-compute[71317]:   <name>instance-00000007</name>
Apr 14 04:16:51 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:16:51 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:16:51 user nova-compute[71317]:   <metadata>
Apr 14 04:16:51 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:16:51 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:16:51 user nova-compute[71317]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-1602894262</nova:name>
Apr 14 04:16:51 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:16:51</nova:creationTime>
Apr 14 04:16:51 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:16:51 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:16:51 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:16:51 user nova-compute[71317]:       <nova:owner>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:user uuid="5234914aec8e443b8a38ac6b38729031">tempest-ServerRescueNegativeTestJSON-508041341-project-member</nova:user>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:project uuid="33320dc2f2c14ecfa5abf6574029c36c">tempest-ServerRescueNegativeTestJSON-508041341</nova:project>
Apr 14 04:16:51 user nova-compute[71317]:       </nova:owner>
Apr 14 04:16:51 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:16:51 user nova-compute[71317]:       <nova:ports>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:port uuid="1201ab18-d7ef-4a92-a77e-d0ab69858538">
Apr 14 04:16:51 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Apr 14 04:16:51 user nova-compute[71317]:         </nova:port>
Apr 14 04:16:51 user nova-compute[71317]:       </nova:ports>
Apr 14 04:16:51 user nova-compute[71317]:     </nova:instance>
Apr 14 04:16:51 user nova-compute[71317]:   </metadata>
Apr 14 04:16:51 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:16:51 user nova-compute[71317]:     <system>
Apr 14 04:16:51 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:16:51 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:16:51 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:16:51 user nova-compute[71317]:       <entry name="serial">2f7e20cd-33ef-4f83-bcdb-a13b9b51370a</entry>
Apr 14 04:16:51 user nova-compute[71317]:       <entry name="uuid">2f7e20cd-33ef-4f83-bcdb-a13b9b51370a</entry>
Apr 14 04:16:51 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:16:51 user nova-compute[71317]:     </system>
Apr 14 04:16:51 user nova-compute[71317]:   </sysinfo>
Apr 14 04:16:51 user nova-compute[71317]:   <os>
Apr 14 04:16:51 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:16:51 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:16:51 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:16:51 user nova-compute[71317]:   </os>
Apr 14 04:16:51 user nova-compute[71317]:   <features>
Apr 14 04:16:51 user nova-compute[71317]:     <acpi/>
Apr 14 04:16:51 user nova-compute[71317]:     <apic/>
Apr 14 04:16:51 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:16:51 user nova-compute[71317]:   </features>
Apr 14 04:16:51 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:16:51 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:16:51 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:16:51 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:16:51 user nova-compute[71317]:   </clock>
Apr 14 04:16:51 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:16:51 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:16:51 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:16:51 user nova-compute[71317]:   </cpu>
Apr 14 04:16:51 user nova-compute[71317]:   <devices>
Apr 14 04:16:51 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:16:51 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:16:51 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk"/>
Apr 14 04:16:51 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:16:51 user nova-compute[71317]:     </disk>
Apr 14 04:16:51 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:16:51 user nova-compute[71317]:       <mac address="fa:16:3e:f5:07:c5"/>
Apr 14 04:16:51 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:51 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:16:51 user nova-compute[71317]:       <target dev="tap1201ab18-d7"/>
Apr 14 04:16:51 user nova-compute[71317]:     </interface>
Apr 14 04:16:51 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:16:51 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/console.log" append="off"/>
Apr 14 04:16:51 user nova-compute[71317]:     </serial>
Apr 14 04:16:51 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:16:51 user nova-compute[71317]:     <video>
Apr 14 04:16:51 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:51 user nova-compute[71317]:     </video>
Apr 14 04:16:51 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:16:51 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:16:51 user nova-compute[71317]:     </rng>
Apr 14 04:16:51 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:16:51 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:16:51 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:16:51 user nova-compute[71317]:     </memballoon>
Apr 14 04:16:51 user nova-compute[71317]:   </devices>
Apr 14 04:16:51 user nova-compute[71317]: </domain>
Apr 14 04:16:51 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1602894262',display_name='tempest-ServerRescueNegativeTestJSON-server-1602894262',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1602894262',id=7,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='33320dc2f2c14ecfa5abf6574029c36c',ramdisk_id='',reservation_id='r-5xk8owm0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-508041341',owner_user_name='tempest-ServerRescueNegativeTestJSON-508041341-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:46Z,user_data=None,user_id='5234914aec8e443b8a38ac6b38729031',uuid=2f7e20cd-33ef-4f83-bcdb-a13b9b51370a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converting VIF {"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f5:07:c5,bridge_name='br-int',has_traffic_filtering=True,id=1201ab18-d7ef-4a92-a77e-d0ab69858538,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1201ab18-d7') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG os_vif [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f5:07:c5,bridge_name='br-int',has_traffic_filtering=True,id=1201ab18-d7ef-4a92-a77e-d0ab69858538,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1201ab18-d7') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap1201ab18-d7, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap1201ab18-d7, col_values=(('external_ids', {'iface-id': '1201ab18-d7ef-4a92-a77e-d0ab69858538', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f5:07:c5', 'vm-uuid': '2f7e20cd-33ef-4f83-bcdb-a13b9b51370a'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:16:51 user nova-compute[71317]: INFO os_vif [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f5:07:c5,bridge_name='br-int',has_traffic_filtering=True,id=1201ab18-d7ef-4a92-a77e-d0ab69858538,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1201ab18-d7')
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.network.neutron [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Updating instance_info_cache with network_info: [{"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Releasing lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Instance network_info: |[{"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Start _get_guest_xml network_info=[{"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No VIF found with MAC fa:16:3e:f5:07:c5, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:16:51 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:51 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1529931408',display_name='tempest-ServerRescueNegativeTestJSON-server-1529931408',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1529931408',id=5,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='33320dc2f2c14ecfa5abf6574029c36c',ramdisk_id='',reservation_id='r-wldnu0b2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-508041341',owner_user_name='tempest-ServerRescueNegativeTestJSON-508041341-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:43Z,user_data=None,user_id='5234914aec8e443b8a38ac6b38729031',uuid=586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converting VIF {"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fb:7a:65,bridge_name='br-int',has_traffic_filtering=True,id=593a3298-746a-48af-96dc-4c572e0d59c2,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap593a3298-74') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'pci_devices' on Instance uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:16:51 user nova-compute[71317]:   <uuid>586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc</uuid>
Apr 14 04:16:51 user nova-compute[71317]:   <name>instance-00000005</name>
Apr 14 04:16:51 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:16:51 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:16:51 user nova-compute[71317]:   <metadata>
Apr 14 04:16:51 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:16:51 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:16:51 user nova-compute[71317]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-1529931408</nova:name>
Apr 14 04:16:51 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:16:51</nova:creationTime>
Apr 14 04:16:51 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:16:51 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:16:51 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:16:51 user nova-compute[71317]:       <nova:owner>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:user uuid="5234914aec8e443b8a38ac6b38729031">tempest-ServerRescueNegativeTestJSON-508041341-project-member</nova:user>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:project uuid="33320dc2f2c14ecfa5abf6574029c36c">tempest-ServerRescueNegativeTestJSON-508041341</nova:project>
Apr 14 04:16:51 user nova-compute[71317]:       </nova:owner>
Apr 14 04:16:51 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:16:51 user nova-compute[71317]:       <nova:ports>
Apr 14 04:16:51 user nova-compute[71317]:         <nova:port uuid="593a3298-746a-48af-96dc-4c572e0d59c2">
Apr 14 04:16:51 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 04:16:51 user nova-compute[71317]:         </nova:port>
Apr 14 04:16:51 user nova-compute[71317]:       </nova:ports>
Apr 14 04:16:51 user nova-compute[71317]:     </nova:instance>
Apr 14 04:16:51 user nova-compute[71317]:   </metadata>
Apr 14 04:16:51 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:16:51 user nova-compute[71317]:     <system>
Apr 14 04:16:51 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:16:51 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:16:51 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:16:51 user nova-compute[71317]:       <entry name="serial">586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc</entry>
Apr 14 04:16:51 user nova-compute[71317]:       <entry name="uuid">586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc</entry>
Apr 14 04:16:51 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:16:51 user nova-compute[71317]:     </system>
Apr 14 04:16:51 user nova-compute[71317]:   </sysinfo>
Apr 14 04:16:51 user nova-compute[71317]:   <os>
Apr 14 04:16:51 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:16:51 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:16:51 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:16:51 user nova-compute[71317]:   </os>
Apr 14 04:16:51 user nova-compute[71317]:   <features>
Apr 14 04:16:51 user nova-compute[71317]:     <acpi/>
Apr 14 04:16:51 user nova-compute[71317]:     <apic/>
Apr 14 04:16:51 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:16:51 user nova-compute[71317]:   </features>
Apr 14 04:16:51 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:16:51 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:16:51 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:16:51 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:16:51 user nova-compute[71317]:   </clock>
Apr 14 04:16:51 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:16:51 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:16:51 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:16:51 user nova-compute[71317]:   </cpu>
Apr 14 04:16:51 user nova-compute[71317]:   <devices>
Apr 14 04:16:51 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:16:51 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:16:51 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk"/>
Apr 14 04:16:51 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:16:51 user nova-compute[71317]:     </disk>
Apr 14 04:16:51 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:16:51 user nova-compute[71317]:       <mac address="fa:16:3e:fb:7a:65"/>
Apr 14 04:16:51 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:51 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:16:51 user nova-compute[71317]:       <target dev="tap593a3298-74"/>
Apr 14 04:16:51 user nova-compute[71317]:     </interface>
Apr 14 04:16:51 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:16:51 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/console.log" append="off"/>
Apr 14 04:16:51 user nova-compute[71317]:     </serial>
Apr 14 04:16:51 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:16:51 user nova-compute[71317]:     <video>
Apr 14 04:16:51 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:51 user nova-compute[71317]:     </video>
Apr 14 04:16:51 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:16:51 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:16:51 user nova-compute[71317]:     </rng>
Apr 14 04:16:51 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:16:51 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:16:51 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:16:51 user nova-compute[71317]:     </memballoon>
Apr 14 04:16:51 user nova-compute[71317]:   </devices>
Apr 14 04:16:51 user nova-compute[71317]: </domain>
Apr 14 04:16:51 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1529931408',display_name='tempest-ServerRescueNegativeTestJSON-server-1529931408',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1529931408',id=5,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='33320dc2f2c14ecfa5abf6574029c36c',ramdisk_id='',reservation_id='r-wldnu0b2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-508041341',owner_user_name='tempest-ServerRescueNegativeTestJSON-508041341-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:43Z,user_data=None,user_id='5234914aec8e443b8a38ac6b38729031',uuid=586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converting VIF {"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fb:7a:65,bridge_name='br-int',has_traffic_filtering=True,id=593a3298-746a-48af-96dc-4c572e0d59c2,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap593a3298-74') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG os_vif [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:7a:65,bridge_name='br-int',has_traffic_filtering=True,id=593a3298-746a-48af-96dc-4c572e0d59c2,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap593a3298-74') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap593a3298-74, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap593a3298-74, col_values=(('external_ids', {'iface-id': '593a3298-746a-48af-96dc-4c572e0d59c2', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:fb:7a:65', 'vm-uuid': '586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:52 user nova-compute[71317]: INFO os_vif [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:7a:65,bridge_name='br-int',has_traffic_filtering=True,id=593a3298-746a-48af-96dc-4c572e0d59c2,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap593a3298-74')
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No VIF found with MAC fa:16:3e:fb:7a:65, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating instance_info_cache with network_info: [{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445812.3642354, e37e27b4-0813-4df9-ac66-1516334c12cb => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:16:52 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] VM Resumed (Lifecycle Event)
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Releasing lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance network_info: |[{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Start _get_guest_xml network_info=[{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:16:52 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Instance spawned successfully.
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:16:52 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:52 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1350614351',display_name='tempest-ServerActionsTestJSON-server-1350614351',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1350614351',id=6,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-789bnjb3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=a0a20aeb-ed06-4c65-b87d-74f8f825457d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.objects.instance [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'pci_devices' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:52 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445812.3695197, e37e27b4-0813-4df9-ac66-1516334c12cb => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:16:52 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] VM Started (Lifecycle Event)
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-changed-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Refreshing instance network info cache due to event network-changed-593a3298-746a-48af-96dc-4c572e0d59c2. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] Acquiring lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] Acquired lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.network.neutron [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Refreshing network info cache for port 593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:16:52 user nova-compute[71317]:   <uuid>a0a20aeb-ed06-4c65-b87d-74f8f825457d</uuid>
Apr 14 04:16:52 user nova-compute[71317]:   <name>instance-00000006</name>
Apr 14 04:16:52 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:16:52 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:16:52 user nova-compute[71317]:   <metadata>
Apr 14 04:16:52 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:16:52 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:16:52 user nova-compute[71317]:       <nova:name>tempest-ServerActionsTestJSON-server-1350614351</nova:name>
Apr 14 04:16:52 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:16:52</nova:creationTime>
Apr 14 04:16:52 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:16:52 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:16:52 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:16:52 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:16:52 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:16:52 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:16:52 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:16:52 user nova-compute[71317]:       <nova:owner>
Apr 14 04:16:52 user nova-compute[71317]:         <nova:user uuid="e35b3ae371cf4dd4b21e2999e58cd2f4">tempest-ServerActionsTestJSON-1048786641-project-member</nova:user>
Apr 14 04:16:52 user nova-compute[71317]:         <nova:project uuid="8c14c332dfcc44b49ddf3cb482dd0d4c">tempest-ServerActionsTestJSON-1048786641</nova:project>
Apr 14 04:16:52 user nova-compute[71317]:       </nova:owner>
Apr 14 04:16:52 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:16:52 user nova-compute[71317]:       <nova:ports>
Apr 14 04:16:52 user nova-compute[71317]:         <nova:port uuid="3a3e3b13-7b4e-46c8-9123-5c1f121a38b9">
Apr 14 04:16:52 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 04:16:52 user nova-compute[71317]:         </nova:port>
Apr 14 04:16:52 user nova-compute[71317]:       </nova:ports>
Apr 14 04:16:52 user nova-compute[71317]:     </nova:instance>
Apr 14 04:16:52 user nova-compute[71317]:   </metadata>
Apr 14 04:16:52 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:16:52 user nova-compute[71317]:     <system>
Apr 14 04:16:52 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:16:52 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:16:52 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:16:52 user nova-compute[71317]:       <entry name="serial">a0a20aeb-ed06-4c65-b87d-74f8f825457d</entry>
Apr 14 04:16:52 user nova-compute[71317]:       <entry name="uuid">a0a20aeb-ed06-4c65-b87d-74f8f825457d</entry>
Apr 14 04:16:52 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:16:52 user nova-compute[71317]:     </system>
Apr 14 04:16:52 user nova-compute[71317]:   </sysinfo>
Apr 14 04:16:52 user nova-compute[71317]:   <os>
Apr 14 04:16:52 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:16:52 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:16:52 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:16:52 user nova-compute[71317]:   </os>
Apr 14 04:16:52 user nova-compute[71317]:   <features>
Apr 14 04:16:52 user nova-compute[71317]:     <acpi/>
Apr 14 04:16:52 user nova-compute[71317]:     <apic/>
Apr 14 04:16:52 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:16:52 user nova-compute[71317]:   </features>
Apr 14 04:16:52 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:16:52 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:16:52 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:16:52 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:16:52 user nova-compute[71317]:   </clock>
Apr 14 04:16:52 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:16:52 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:16:52 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:16:52 user nova-compute[71317]:   </cpu>
Apr 14 04:16:52 user nova-compute[71317]:   <devices>
Apr 14 04:16:52 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:16:52 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:16:52 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk"/>
Apr 14 04:16:52 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:16:52 user nova-compute[71317]:     </disk>
Apr 14 04:16:52 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:16:52 user nova-compute[71317]:       <mac address="fa:16:3e:5d:8f:83"/>
Apr 14 04:16:52 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:52 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:16:52 user nova-compute[71317]:       <target dev="tap3a3e3b13-7b"/>
Apr 14 04:16:52 user nova-compute[71317]:     </interface>
Apr 14 04:16:52 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:16:52 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/console.log" append="off"/>
Apr 14 04:16:52 user nova-compute[71317]:     </serial>
Apr 14 04:16:52 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:16:52 user nova-compute[71317]:     <video>
Apr 14 04:16:52 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:52 user nova-compute[71317]:     </video>
Apr 14 04:16:52 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:16:52 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:16:52 user nova-compute[71317]:     </rng>
Apr 14 04:16:52 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:16:52 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:16:52 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:16:52 user nova-compute[71317]:     </memballoon>
Apr 14 04:16:52 user nova-compute[71317]:   </devices>
Apr 14 04:16:52 user nova-compute[71317]: </domain>
Apr 14 04:16:52 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1350614351',display_name='tempest-ServerActionsTestJSON-server-1350614351',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1350614351',id=6,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-789bnjb3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=a0a20aeb-ed06-4c65-b87d-74f8f825457d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG os_vif [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3a3e3b13-7b, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap3a3e3b13-7b, col_values=(('external_ids', {'iface-id': '3a3e3b13-7b4e-46c8-9123-5c1f121a38b9', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:5d:8f:83', 'vm-uuid': 'a0a20aeb-ed06-4c65-b87d-74f8f825457d'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:16:52 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:16:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:52 user nova-compute[71317]: INFO os_vif [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b')
Apr 14 04:16:52 user nova-compute[71317]: INFO nova.compute.manager [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Took 13.72 seconds to spawn the instance on the hypervisor.
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No VIF found with MAC fa:16:3e:5d:8f:83, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received event network-changed-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Refreshing instance network info cache due to event network-changed-1201ab18-d7ef-4a92-a77e-d0ab69858538. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] Acquiring lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] Acquired lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:52 user nova-compute[71317]: DEBUG nova.network.neutron [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Refreshing network info cache for port 1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:16:52 user nova-compute[71317]: INFO nova.compute.manager [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Took 14.65 seconds to build instance.
Apr 14 04:16:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-47e05cd5-3805-4bd5-b160-70cae4a76836 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 14.768s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Updated VIF entry in instance network info cache for port 593a3298-746a-48af-96dc-4c572e0d59c2. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Updating instance_info_cache with network_info: [{"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] Releasing lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-changed-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Refreshing instance network info cache due to event network-changed-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] Acquiring lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] Acquired lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Refreshing network info cache for port 3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Updated VIF entry in instance network info cache for port 1201ab18-d7ef-4a92-a77e-d0ab69858538. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Updating instance_info_cache with network_info: [{"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] Releasing lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Received event network-vif-plugged-2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] Acquiring lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] No waiting events found dispatching network-vif-plugged-2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:54 user nova-compute[71317]: WARNING nova.compute.manager [req-07bd9b15-122a-4ea8-b613-836a874becdf req-8abd6fe9-e685-40aa-be0d-8bbf081e6af7 service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Received unexpected event network-vif-plugged-2a342575-05e3-4179-a370-6d39ac4f815d for instance with vm_state building and task_state spawning.
Apr 14 04:16:54 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Successfully created port: 0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:16:55 user nova-compute[71317]: DEBUG nova.network.neutron [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updated VIF entry in instance network info cache for port 3a3e3b13-7b4e-46c8-9123-5c1f121a38b9. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:16:55 user nova-compute[71317]: DEBUG nova.network.neutron [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating instance_info_cache with network_info: [{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c5d9b6a2-8cef-478c-a775-36fe6d74b6db req-6532d325-6ff7-47a5-bdf3-2b4f32d79afc service nova] Releasing lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-1fb678ce-d735-4117-bedd-d2697e47d467 req-00c73996-c11b-4765-a317-c00010f96cef service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1fb678ce-d735-4117-bedd-d2697e47d467 req-00c73996-c11b-4765-a317-c00010f96cef service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1fb678ce-d735-4117-bedd-d2697e47d467 req-00c73996-c11b-4765-a317-c00010f96cef service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1fb678ce-d735-4117-bedd-d2697e47d467 req-00c73996-c11b-4765-a317-c00010f96cef service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-1fb678ce-d735-4117-bedd-d2697e47d467 req-00c73996-c11b-4765-a317-c00010f96cef service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:57 user nova-compute[71317]: WARNING nova.compute.manager [req-1fb678ce-d735-4117-bedd-d2697e47d467 req-00c73996-c11b-4765-a317-c00010f96cef service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state building and task_state spawning.
Apr 14 04:16:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-3fefd8cf-7541-4c98-a30a-10f493ce999b req-21bef985-711f-4dbb-abb8-64fbb7dad434 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received event network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3fefd8cf-7541-4c98-a30a-10f493ce999b req-21bef985-711f-4dbb-abb8-64fbb7dad434 service nova] Acquiring lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3fefd8cf-7541-4c98-a30a-10f493ce999b req-21bef985-711f-4dbb-abb8-64fbb7dad434 service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3fefd8cf-7541-4c98-a30a-10f493ce999b req-21bef985-711f-4dbb-abb8-64fbb7dad434 service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-3fefd8cf-7541-4c98-a30a-10f493ce999b req-21bef985-711f-4dbb-abb8-64fbb7dad434 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] No waiting events found dispatching network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:57 user nova-compute[71317]: WARNING nova.compute.manager [req-3fefd8cf-7541-4c98-a30a-10f493ce999b req-21bef985-711f-4dbb-abb8-64fbb7dad434 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received unexpected event network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 for instance with vm_state building and task_state spawning.
Apr 14 04:16:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-aa2641e2-d08c-4435-8ed1-b0a7991d3110 req-1fb74761-0180-4270-9c94-2a03851a3a73 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-aa2641e2-d08c-4435-8ed1-b0a7991d3110 req-1fb74761-0180-4270-9c94-2a03851a3a73 service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-aa2641e2-d08c-4435-8ed1-b0a7991d3110 req-1fb74761-0180-4270-9c94-2a03851a3a73 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-aa2641e2-d08c-4435-8ed1-b0a7991d3110 req-1fb74761-0180-4270-9c94-2a03851a3a73 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-aa2641e2-d08c-4435-8ed1-b0a7991d3110 req-1fb74761-0180-4270-9c94-2a03851a3a73 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:57 user nova-compute[71317]: WARNING nova.compute.manager [req-aa2641e2-d08c-4435-8ed1-b0a7991d3110 req-1fb74761-0180-4270-9c94-2a03851a3a73 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state building and task_state spawning.
Apr 14 04:16:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Successfully updated port: 0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:16:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquired lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:16:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:16:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-6830ee71-c32b-4fc3-bec3-6b58a5780c2b req-d94ee161-2b48-44ef-acfa-cd5ffad3ed57 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received event network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6830ee71-c32b-4fc3-bec3-6b58a5780c2b req-d94ee161-2b48-44ef-acfa-cd5ffad3ed57 service nova] Acquiring lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6830ee71-c32b-4fc3-bec3-6b58a5780c2b req-d94ee161-2b48-44ef-acfa-cd5ffad3ed57 service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6830ee71-c32b-4fc3-bec3-6b58a5780c2b req-d94ee161-2b48-44ef-acfa-cd5ffad3ed57 service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-6830ee71-c32b-4fc3-bec3-6b58a5780c2b req-d94ee161-2b48-44ef-acfa-cd5ffad3ed57 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] No waiting events found dispatching network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:59 user nova-compute[71317]: WARNING nova.compute.manager [req-6830ee71-c32b-4fc3-bec3-6b58a5780c2b req-d94ee161-2b48-44ef-acfa-cd5ffad3ed57 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received unexpected event network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 for instance with vm_state building and task_state spawning.
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-d91adc75-c262-46f9-ac99-2dceac2c46b6 req-49d739ba-dffd-41ce-8bd7-f2a4c2f6463a service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d91adc75-c262-46f9-ac99-2dceac2c46b6 req-49d739ba-dffd-41ce-8bd7-f2a4c2f6463a service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d91adc75-c262-46f9-ac99-2dceac2c46b6 req-49d739ba-dffd-41ce-8bd7-f2a4c2f6463a service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d91adc75-c262-46f9-ac99-2dceac2c46b6 req-49d739ba-dffd-41ce-8bd7-f2a4c2f6463a service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-d91adc75-c262-46f9-ac99-2dceac2c46b6 req-49d739ba-dffd-41ce-8bd7-f2a4c2f6463a service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:59 user nova-compute[71317]: WARNING nova.compute.manager [req-d91adc75-c262-46f9-ac99-2dceac2c46b6 req-49d739ba-dffd-41ce-8bd7-f2a4c2f6463a service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state building and task_state spawning.
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updating instance_info_cache with network_info: [{"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Releasing lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Instance network_info: |[{"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Start _get_guest_xml network_info=[{"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:16:59 user nova-compute[71317]: WARNING nova.compute.manager [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state building and task_state spawning.
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-changed-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Refreshing instance network info cache due to event network-changed-0c62956f-a2d8-4932-b48f-469eba99b8e6. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] Acquiring lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] Acquired lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Refreshing network info cache for port 0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:16:59 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:59 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-1335768391',display_name='tempest-ServerStableDeviceRescueTest-server-1335768391',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-1335768391',id=8,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHsERCJfV5vlojYp6CtYNDuskD1mPJwApYPpyopwqnFEg3bukLlc5MC0Mx5e6kcUmmhflLvGI9V/TTXxf//CKjD9nOwYO+5pKfSRGQB3deK57Z7HLLcWkH9v7l2Z+jP7Gg==',key_name='tempest-keypair-234703296',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='15bb9ddec81e4f6cbceb6ce7dee9c3a1',ramdisk_id='',reservation_id='r-9gfjdvk7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerStableDeviceRescueTest-1018370864',owner_user_name='tempest-ServerStableDeviceRescueTest-1018370864-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='676d80d3684045298d895555247174af',uuid=cd60eede-cd79-4c1b-adb7-ba7c6cbc9659,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Converting VIF {"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:73:25:41,bridge_name='br-int',has_traffic_filtering=True,id=0c62956f-a2d8-4932-b48f-469eba99b8e6,network=Network(97d3cce1-27a3-4e76-9d3f-44860a2a8dcf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0c62956f-a2') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.objects.instance [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'pci_devices' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:16:59 user nova-compute[71317]:   <uuid>cd60eede-cd79-4c1b-adb7-ba7c6cbc9659</uuid>
Apr 14 04:16:59 user nova-compute[71317]:   <name>instance-00000008</name>
Apr 14 04:16:59 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:16:59 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:16:59 user nova-compute[71317]:   <metadata>
Apr 14 04:16:59 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:16:59 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:16:59 user nova-compute[71317]:       <nova:name>tempest-ServerStableDeviceRescueTest-server-1335768391</nova:name>
Apr 14 04:16:59 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:16:59</nova:creationTime>
Apr 14 04:16:59 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:16:59 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:16:59 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:16:59 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:16:59 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:16:59 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:16:59 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:16:59 user nova-compute[71317]:       <nova:owner>
Apr 14 04:16:59 user nova-compute[71317]:         <nova:user uuid="676d80d3684045298d895555247174af">tempest-ServerStableDeviceRescueTest-1018370864-project-member</nova:user>
Apr 14 04:16:59 user nova-compute[71317]:         <nova:project uuid="15bb9ddec81e4f6cbceb6ce7dee9c3a1">tempest-ServerStableDeviceRescueTest-1018370864</nova:project>
Apr 14 04:16:59 user nova-compute[71317]:       </nova:owner>
Apr 14 04:16:59 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:16:59 user nova-compute[71317]:       <nova:ports>
Apr 14 04:16:59 user nova-compute[71317]:         <nova:port uuid="0c62956f-a2d8-4932-b48f-469eba99b8e6">
Apr 14 04:16:59 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 04:16:59 user nova-compute[71317]:         </nova:port>
Apr 14 04:16:59 user nova-compute[71317]:       </nova:ports>
Apr 14 04:16:59 user nova-compute[71317]:     </nova:instance>
Apr 14 04:16:59 user nova-compute[71317]:   </metadata>
Apr 14 04:16:59 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:16:59 user nova-compute[71317]:     <system>
Apr 14 04:16:59 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:16:59 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:16:59 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:16:59 user nova-compute[71317]:       <entry name="serial">cd60eede-cd79-4c1b-adb7-ba7c6cbc9659</entry>
Apr 14 04:16:59 user nova-compute[71317]:       <entry name="uuid">cd60eede-cd79-4c1b-adb7-ba7c6cbc9659</entry>
Apr 14 04:16:59 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:16:59 user nova-compute[71317]:     </system>
Apr 14 04:16:59 user nova-compute[71317]:   </sysinfo>
Apr 14 04:16:59 user nova-compute[71317]:   <os>
Apr 14 04:16:59 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:16:59 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:16:59 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:16:59 user nova-compute[71317]:   </os>
Apr 14 04:16:59 user nova-compute[71317]:   <features>
Apr 14 04:16:59 user nova-compute[71317]:     <acpi/>
Apr 14 04:16:59 user nova-compute[71317]:     <apic/>
Apr 14 04:16:59 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:16:59 user nova-compute[71317]:   </features>
Apr 14 04:16:59 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:16:59 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:16:59 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:16:59 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:16:59 user nova-compute[71317]:   </clock>
Apr 14 04:16:59 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:16:59 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:16:59 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:16:59 user nova-compute[71317]:   </cpu>
Apr 14 04:16:59 user nova-compute[71317]:   <devices>
Apr 14 04:16:59 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:16:59 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:16:59 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk"/>
Apr 14 04:16:59 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:16:59 user nova-compute[71317]:     </disk>
Apr 14 04:16:59 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:16:59 user nova-compute[71317]:       <mac address="fa:16:3e:73:25:41"/>
Apr 14 04:16:59 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:59 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:16:59 user nova-compute[71317]:       <target dev="tap0c62956f-a2"/>
Apr 14 04:16:59 user nova-compute[71317]:     </interface>
Apr 14 04:16:59 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:16:59 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/console.log" append="off"/>
Apr 14 04:16:59 user nova-compute[71317]:     </serial>
Apr 14 04:16:59 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:16:59 user nova-compute[71317]:     <video>
Apr 14 04:16:59 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:16:59 user nova-compute[71317]:     </video>
Apr 14 04:16:59 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:16:59 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:16:59 user nova-compute[71317]:     </rng>
Apr 14 04:16:59 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:16:59 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:16:59 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:16:59 user nova-compute[71317]:     </memballoon>
Apr 14 04:16:59 user nova-compute[71317]:   </devices>
Apr 14 04:16:59 user nova-compute[71317]: </domain>
Apr 14 04:16:59 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-1335768391',display_name='tempest-ServerStableDeviceRescueTest-server-1335768391',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-1335768391',id=8,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHsERCJfV5vlojYp6CtYNDuskD1mPJwApYPpyopwqnFEg3bukLlc5MC0Mx5e6kcUmmhflLvGI9V/TTXxf//CKjD9nOwYO+5pKfSRGQB3deK57Z7HLLcWkH9v7l2Z+jP7Gg==',key_name='tempest-keypair-234703296',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='15bb9ddec81e4f6cbceb6ce7dee9c3a1',ramdisk_id='',reservation_id='r-9gfjdvk7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerStableDeviceRescueTest-1018370864',owner_user_name='tempest-ServerStableDeviceRescueTest-1018370864-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:16:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='676d80d3684045298d895555247174af',uuid=cd60eede-cd79-4c1b-adb7-ba7c6cbc9659,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Converting VIF {"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:73:25:41,bridge_name='br-int',has_traffic_filtering=True,id=0c62956f-a2d8-4932-b48f-469eba99b8e6,network=Network(97d3cce1-27a3-4e76-9d3f-44860a2a8dcf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0c62956f-a2') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG os_vif [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:73:25:41,bridge_name='br-int',has_traffic_filtering=True,id=0c62956f-a2d8-4932-b48f-469eba99b8e6,network=Network(97d3cce1-27a3-4e76-9d3f-44860a2a8dcf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0c62956f-a2') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap0c62956f-a2, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap0c62956f-a2, col_values=(('external_ids', {'iface-id': '0c62956f-a2d8-4932-b48f-469eba99b8e6', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:73:25:41', 'vm-uuid': 'cd60eede-cd79-4c1b-adb7-ba7c6cbc9659'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:16:59 user nova-compute[71317]: INFO os_vif [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:73:25:41,bridge_name='br-int',has_traffic_filtering=True,id=0c62956f-a2d8-4932-b48f-469eba99b8e6,network=Network(97d3cce1-27a3-4e76-9d3f-44860a2a8dcf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0c62956f-a2')
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:16:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] No VIF found with MAC fa:16:3e:73:25:41, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445820.179016, 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] VM Resumed (Lifecycle Event)
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Instance spawned successfully.
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Instance spawned successfully.
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445820.1792097, 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] VM Started (Lifecycle Event)
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445820.2147818, 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] VM Resumed (Lifecycle Event)
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance spawned successfully.
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.compute.manager [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Took 17.19 seconds to spawn the instance on the hypervisor.
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.compute.manager [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Took 14.03 seconds to spawn the instance on the hypervisor.
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445820.214944, 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] VM Started (Lifecycle Event)
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:00 user nova-compute[71317]: INFO nova.compute.manager [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Took 18.62 seconds to build instance.
Apr 14 04:17:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ad3c5ec-6f72-465f-af20-cd4bc21c04b4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 18.782s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:01 user nova-compute[71317]: DEBUG nova.network.neutron [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updated VIF entry in instance network info cache for port 0c62956f-a2d8-4932-b48f-469eba99b8e6. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:17:01 user nova-compute[71317]: DEBUG nova.network.neutron [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updating instance_info_cache with network_info: [{"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:01 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:01 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445820.5230308, a0a20aeb-ed06-4c65-b87d-74f8f825457d => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:01 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] VM Resumed (Lifecycle Event)
Apr 14 04:17:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2c397b30-725b-46d8-b84c-ca32b1de8609 req-3437ffb9-375e-4f39-a333-820acefb2b24 service nova] Releasing lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:01 user nova-compute[71317]: INFO nova.compute.manager [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Took 17.12 seconds to spawn the instance on the hypervisor.
Apr 14 04:17:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:01 user nova-compute[71317]: INFO nova.compute.manager [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Took 15.86 seconds to build instance.
Apr 14 04:17:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-da85801e-a6ab-44ca-aed3-ba6fbea0c1ab tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 15.997s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:01 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:01 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445820.5311532, a0a20aeb-ed06-4c65-b87d-74f8f825457d => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:01 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] VM Started (Lifecycle Event)
Apr 14 04:17:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:01 user nova-compute[71317]: INFO nova.compute.manager [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Took 18.55 seconds to build instance.
Apr 14 04:17:01 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-322c274f-4d8b-4248-9252-4b8acab0183b tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 18.766s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-63f80013-44c1-4353-b245-689d62c2d058 req-917cfce3-966e-414b-8587-272056081711 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-63f80013-44c1-4353-b245-689d62c2d058 req-917cfce3-966e-414b-8587-272056081711 service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-63f80013-44c1-4353-b245-689d62c2d058 req-917cfce3-966e-414b-8587-272056081711 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-63f80013-44c1-4353-b245-689d62c2d058 req-917cfce3-966e-414b-8587-272056081711 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-63f80013-44c1-4353-b245-689d62c2d058 req-917cfce3-966e-414b-8587-272056081711 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:17:02 user nova-compute[71317]: WARNING nova.compute.manager [req-63f80013-44c1-4353-b245-689d62c2d058 req-917cfce3-966e-414b-8587-272056081711 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state building and task_state spawning.
Apr 14 04:17:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:17:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:17:03 user nova-compute[71317]: INFO nova.compute.claims [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Claim successful on node user
Apr 14 04:17:04 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.857s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG nova.network.neutron [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:17:04 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Ignoring supplied device name: /dev/sda. Libvirt can't honour user-supplied dev names
Apr 14 04:17:04 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG nova.policy [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd1b09bc7a0344ae782d33d3222a8ea88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0e636ba0ee9486f9f48905d886ab547', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:17:04 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Creating image(s)
Apr 14 04:17:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "/opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "/opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "/opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "be368c247365fec2f9d8f8c8897c09b99127635e" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "be368c247365fec2f9d8f8c8897c09b99127635e" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-af719bcc-9105-43f6-bd79-55c6e2fd7969 req-43c05b80-8877-4c22-b2b2-c19a2f344ef6 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-af719bcc-9105-43f6-bd79-55c6e2fd7969 req-43c05b80-8877-4c22-b2b2-c19a2f344ef6 service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-af719bcc-9105-43f6-bd79-55c6e2fd7969 req-43c05b80-8877-4c22-b2b2-c19a2f344ef6 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-af719bcc-9105-43f6-bd79-55c6e2fd7969 req-43c05b80-8877-4c22-b2b2-c19a2f344ef6 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-af719bcc-9105-43f6-bd79-55c6e2fd7969 req-43c05b80-8877-4c22-b2b2-c19a2f344ef6 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:17:04 user nova-compute[71317]: WARNING nova.compute.manager [req-af719bcc-9105-43f6-bd79-55c6e2fd7969 req-43c05b80-8877-4c22-b2b2-c19a2f344ef6 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state building and task_state spawning.
Apr 14 04:17:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e.part --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:05 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445825.2608554, cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:05 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] VM Resumed (Lifecycle Event)
Apr 14 04:17:05 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:17:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:17:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:05 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:05 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Instance spawned successfully.
Apr 14 04:17:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:06 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445825.2658508, cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:06 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] VM Started (Lifecycle Event)
Apr 14 04:17:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e.part --force-share --output=json" returned: 0 in 0.377s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.virt.images [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] f478712d-cb56-468b-a1b1-14aa5eab7104 was qcow2, converting to raw {{(pid=71317) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e.part /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e.converted {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:06 user nova-compute[71317]: INFO nova.compute.manager [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Took 16.89 seconds to spawn the instance on the hypervisor.
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:17:06 user nova-compute[71317]: INFO nova.compute.claims [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Claim successful on node user
Apr 14 04:17:06 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:06 user nova-compute[71317]: INFO nova.compute.manager [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Took 18.02 seconds to build instance.
Apr 14 04:17:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fd89ffb3-188b-4f85-8f59-d28932ea04dc tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 18.187s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e.part /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e.converted" returned: 0 in 0.424s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e.converted --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG nova.network.neutron [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Successfully created port: 2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e.converted --force-share --output=json" returned: 0 in 0.265s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "be368c247365fec2f9d8f8c8897c09b99127635e" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 2.031s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e --force-share --output=json" returned: 0 in 0.169s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "be368c247365fec2f9d8f8c8897c09b99127635e" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "be368c247365fec2f9d8f8c8897c09b99127635e" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.895s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e --force-share --output=json" returned: 0 in 0.199s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e,backing_fmt=raw /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e,backing_fmt=raw /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk 1073741824" returned: 0 in 0.074s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "be368c247365fec2f9d8f8c8897c09b99127635e" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.288s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.network.neutron [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:17:07 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.network.neutron [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Successfully updated port: 2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e --force-share --output=json" returned: 0 in 0.255s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Checking if we can resize image /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.policy [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0aeeeae7cbc340daa8cd81c1418794e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01cefa66350a4432ad71204b39e953be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "refresh_cache-e5344b0d-2d73-40c7-8ac3-535396e985a4" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquired lock "refresh_cache-e5344b0d-2d73-40c7-8ac3-535396e985a4" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.network.neutron [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-a9c3c8a1-195b-48a2-b871-3fa09d0a5629 req-49e00a46-01eb-4850-a131-78d2ea502762 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Received event network-changed-2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-a9c3c8a1-195b-48a2-b871-3fa09d0a5629 req-49e00a46-01eb-4850-a131-78d2ea502762 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Refreshing instance network info cache due to event network-changed-2d24208b-94ce-4364-86a4-1758c24e9047. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a9c3c8a1-195b-48a2-b871-3fa09d0a5629 req-49e00a46-01eb-4850-a131-78d2ea502762 service nova] Acquiring lock "refresh_cache-e5344b0d-2d73-40c7-8ac3-535396e985a4" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:17:07 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Creating image(s)
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.network.neutron [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk --force-share --output=json" returned: 0 in 0.200s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Cannot resize image /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.objects.instance [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lazy-loading 'migration_context' on Instance uuid e5344b0d-2d73-40c7-8ac3-535396e985a4 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Ensure instance console log exists: /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.204s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.201s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.network.neutron [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Updating instance_info_cache with network_info: [{"id": "2d24208b-94ce-4364-86a4-1758c24e9047", "address": "fa:16:3e:22:c7:b3", "network": {"id": "69963625-cff2-4260-908f-82359d8b2f77", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-558617934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e0e636ba0ee9486f9f48905d886ab547", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d24208b-94", "ovs_interfaceid": "2d24208b-94ce-4364-86a4-1758c24e9047", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk 1073741824" returned: 0 in 0.072s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.282s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Releasing lock "refresh_cache-e5344b0d-2d73-40c7-8ac3-535396e985a4" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Instance network_info: |[{"id": "2d24208b-94ce-4364-86a4-1758c24e9047", "address": "fa:16:3e:22:c7:b3", "network": {"id": "69963625-cff2-4260-908f-82359d8b2f77", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-558617934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e0e636ba0ee9486f9f48905d886ab547", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d24208b-94", "ovs_interfaceid": "2d24208b-94ce-4364-86a4-1758c24e9047", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a9c3c8a1-195b-48a2-b871-3fa09d0a5629 req-49e00a46-01eb-4850-a131-78d2ea502762 service nova] Acquired lock "refresh_cache-e5344b0d-2d73-40c7-8ac3-535396e985a4" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.network.neutron [req-a9c3c8a1-195b-48a2-b871-3fa09d0a5629 req-49e00a46-01eb-4850-a131-78d2ea502762 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Refreshing network info cache for port 2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Start _get_guest_xml network_info=[{"id": "2d24208b-94ce-4364-86a4-1758c24e9047", "address": "fa:16:3e:22:c7:b3", "network": {"id": "69963625-cff2-4260-908f-82359d8b2f77", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-558617934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e0e636ba0ee9486f9f48905d886ab547", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d24208b-94", "ovs_interfaceid": "2d24208b-94ce-4364-86a4-1758c24e9047", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'scsi', 'cdrom_bus': 'scsi', 'mapping': {'root': {'bus': 'scsi', 'dev': 'sda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'scsi', 'dev': 'sda', 'type': 'disk', 'boot_index': '1'}, 'disk.config': {'bus': 'scsi', 'dev': 'sdb', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:16:53Z,direct_url=<?>,disk_format='qcow2',id=f478712d-cb56-468b-a1b1-14aa5eab7104,min_disk=0,min_ram=0,name='',owner='2ac2150930fe4f8a94005ca330685231',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:16:55Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/sda', 'image': [{'disk_bus': 'scsi', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/sda', 'image_id': 'f478712d-cb56-468b-a1b1-14aa5eab7104'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:17:08 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:17:08 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:16:53Z,direct_url=<?>,disk_format='qcow2',id=f478712d-cb56-468b-a1b1-14aa5eab7104,min_disk=0,min_ram=0,name='',owner='2ac2150930fe4f8a94005ca330685231',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:16:55Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-1544950554',display_name='tempest-AttachSCSIVolumeTestJSON-server-1544950554',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-1544950554',id=9,image_ref='f478712d-cb56-468b-a1b1-14aa5eab7104',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP2a/q8z148VuqmzCdTppqcm7Q47cCuI3B0oR8ww+C0FRQKet9rnqUXrh5enWkvTgHjdhAOOgHwD+H7zMvAp4yzILDVjHd5SWnmFQ8IxBan80X7GvaW+XLPf39HLZ/j3xA==',key_name='tempest-keypair-2075478387',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e0e636ba0ee9486f9f48905d886ab547',ramdisk_id='',reservation_id='r-vkwun6nu',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='f478712d-cb56-468b-a1b1-14aa5eab7104',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_machine_type='pc',image_hw_scsi_model='virtio-scsi',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-AttachSCSIVolumeTestJSON-1503158328',owner_user_name='tempest-AttachSCSIVolumeTestJSON-1503158328-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:05Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d1b09bc7a0344ae782d33d3222a8ea88',uuid=e5344b0d-2d73-40c7-8ac3-535396e985a4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2d24208b-94ce-4364-86a4-1758c24e9047", "address": "fa:16:3e:22:c7:b3", "network": {"id": "69963625-cff2-4260-908f-82359d8b2f77", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-558617934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e0e636ba0ee9486f9f48905d886ab547", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d24208b-94", "ovs_interfaceid": "2d24208b-94ce-4364-86a4-1758c24e9047", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Converting VIF {"id": "2d24208b-94ce-4364-86a4-1758c24e9047", "address": "fa:16:3e:22:c7:b3", "network": {"id": "69963625-cff2-4260-908f-82359d8b2f77", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-558617934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e0e636ba0ee9486f9f48905d886ab547", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d24208b-94", "ovs_interfaceid": "2d24208b-94ce-4364-86a4-1758c24e9047", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:22:c7:b3,bridge_name='br-int',has_traffic_filtering=True,id=2d24208b-94ce-4364-86a4-1758c24e9047,network=Network(69963625-cff2-4260-908f-82359d8b2f77),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d24208b-94') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.objects.instance [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lazy-loading 'pci_devices' on Instance uuid e5344b0d-2d73-40c7-8ac3-535396e985a4 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.177s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Checking if we can resize image /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:17:08 user nova-compute[71317]:   <uuid>e5344b0d-2d73-40c7-8ac3-535396e985a4</uuid>
Apr 14 04:17:08 user nova-compute[71317]:   <name>instance-00000009</name>
Apr 14 04:17:08 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:17:08 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:17:08 user nova-compute[71317]:   <metadata>
Apr 14 04:17:08 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:17:08 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:17:08 user nova-compute[71317]:       <nova:name>tempest-AttachSCSIVolumeTestJSON-server-1544950554</nova:name>
Apr 14 04:17:08 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:17:08</nova:creationTime>
Apr 14 04:17:08 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:17:08 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:17:08 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:17:08 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:17:08 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:17:08 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:17:08 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:17:08 user nova-compute[71317]:       <nova:owner>
Apr 14 04:17:08 user nova-compute[71317]:         <nova:user uuid="d1b09bc7a0344ae782d33d3222a8ea88">tempest-AttachSCSIVolumeTestJSON-1503158328-project-member</nova:user>
Apr 14 04:17:08 user nova-compute[71317]:         <nova:project uuid="e0e636ba0ee9486f9f48905d886ab547">tempest-AttachSCSIVolumeTestJSON-1503158328</nova:project>
Apr 14 04:17:08 user nova-compute[71317]:       </nova:owner>
Apr 14 04:17:08 user nova-compute[71317]:       <nova:root type="image" uuid="f478712d-cb56-468b-a1b1-14aa5eab7104"/>
Apr 14 04:17:08 user nova-compute[71317]:       <nova:ports>
Apr 14 04:17:08 user nova-compute[71317]:         <nova:port uuid="2d24208b-94ce-4364-86a4-1758c24e9047">
Apr 14 04:17:08 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 04:17:08 user nova-compute[71317]:         </nova:port>
Apr 14 04:17:08 user nova-compute[71317]:       </nova:ports>
Apr 14 04:17:08 user nova-compute[71317]:     </nova:instance>
Apr 14 04:17:08 user nova-compute[71317]:   </metadata>
Apr 14 04:17:08 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:17:08 user nova-compute[71317]:     <system>
Apr 14 04:17:08 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:17:08 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:17:08 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:17:08 user nova-compute[71317]:       <entry name="serial">e5344b0d-2d73-40c7-8ac3-535396e985a4</entry>
Apr 14 04:17:08 user nova-compute[71317]:       <entry name="uuid">e5344b0d-2d73-40c7-8ac3-535396e985a4</entry>
Apr 14 04:17:08 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:17:08 user nova-compute[71317]:     </system>
Apr 14 04:17:08 user nova-compute[71317]:   </sysinfo>
Apr 14 04:17:08 user nova-compute[71317]:   <os>
Apr 14 04:17:08 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:17:08 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:17:08 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:17:08 user nova-compute[71317]:   </os>
Apr 14 04:17:08 user nova-compute[71317]:   <features>
Apr 14 04:17:08 user nova-compute[71317]:     <acpi/>
Apr 14 04:17:08 user nova-compute[71317]:     <apic/>
Apr 14 04:17:08 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:17:08 user nova-compute[71317]:   </features>
Apr 14 04:17:08 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:17:08 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:17:08 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:17:08 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:17:08 user nova-compute[71317]:   </clock>
Apr 14 04:17:08 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:17:08 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:17:08 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:17:08 user nova-compute[71317]:   </cpu>
Apr 14 04:17:08 user nova-compute[71317]:   <devices>
Apr 14 04:17:08 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:17:08 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:17:08 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk"/>
Apr 14 04:17:08 user nova-compute[71317]:       <target dev="sda" bus="scsi"/>
Apr 14 04:17:08 user nova-compute[71317]:       <address type="drive" controller="0" unit="0"/>
Apr 14 04:17:08 user nova-compute[71317]:     </disk>
Apr 14 04:17:08 user nova-compute[71317]:     <disk type="file" device="cdrom">
Apr 14 04:17:08 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none"/>
Apr 14 04:17:08 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk.config"/>
Apr 14 04:17:08 user nova-compute[71317]:       <target dev="sdb" bus="scsi"/>
Apr 14 04:17:08 user nova-compute[71317]:       <address type="drive" controller="0" unit="1"/>
Apr 14 04:17:08 user nova-compute[71317]:     </disk>
Apr 14 04:17:08 user nova-compute[71317]:     <controller type="scsi" index="0" model="virtio-scsi"/>
Apr 14 04:17:08 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:17:08 user nova-compute[71317]:       <mac address="fa:16:3e:22:c7:b3"/>
Apr 14 04:17:08 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:17:08 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:17:08 user nova-compute[71317]:       <target dev="tap2d24208b-94"/>
Apr 14 04:17:08 user nova-compute[71317]:     </interface>
Apr 14 04:17:08 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:17:08 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/console.log" append="off"/>
Apr 14 04:17:08 user nova-compute[71317]:     </serial>
Apr 14 04:17:08 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:17:08 user nova-compute[71317]:     <video>
Apr 14 04:17:08 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:17:08 user nova-compute[71317]:     </video>
Apr 14 04:17:08 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:17:08 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:17:08 user nova-compute[71317]:     </rng>
Apr 14 04:17:08 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:17:08 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:17:08 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:17:08 user nova-compute[71317]:     </memballoon>
Apr 14 04:17:08 user nova-compute[71317]:   </devices>
Apr 14 04:17:08 user nova-compute[71317]: </domain>
Apr 14 04:17:08 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-1544950554',display_name='tempest-AttachSCSIVolumeTestJSON-server-1544950554',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-1544950554',id=9,image_ref='f478712d-cb56-468b-a1b1-14aa5eab7104',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP2a/q8z148VuqmzCdTppqcm7Q47cCuI3B0oR8ww+C0FRQKet9rnqUXrh5enWkvTgHjdhAOOgHwD+H7zMvAp4yzILDVjHd5SWnmFQ8IxBan80X7GvaW+XLPf39HLZ/j3xA==',key_name='tempest-keypair-2075478387',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e0e636ba0ee9486f9f48905d886ab547',ramdisk_id='',reservation_id='r-vkwun6nu',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='f478712d-cb56-468b-a1b1-14aa5eab7104',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_machine_type='pc',image_hw_scsi_model='virtio-scsi',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-AttachSCSIVolumeTestJSON-1503158328',owner_user_name='tempest-AttachSCSIVolumeTestJSON-1503158328-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:05Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d1b09bc7a0344ae782d33d3222a8ea88',uuid=e5344b0d-2d73-40c7-8ac3-535396e985a4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2d24208b-94ce-4364-86a4-1758c24e9047", "address": "fa:16:3e:22:c7:b3", "network": {"id": "69963625-cff2-4260-908f-82359d8b2f77", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-558617934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e0e636ba0ee9486f9f48905d886ab547", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d24208b-94", "ovs_interfaceid": "2d24208b-94ce-4364-86a4-1758c24e9047", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Converting VIF {"id": "2d24208b-94ce-4364-86a4-1758c24e9047", "address": "fa:16:3e:22:c7:b3", "network": {"id": "69963625-cff2-4260-908f-82359d8b2f77", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-558617934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e0e636ba0ee9486f9f48905d886ab547", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d24208b-94", "ovs_interfaceid": "2d24208b-94ce-4364-86a4-1758c24e9047", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:22:c7:b3,bridge_name='br-int',has_traffic_filtering=True,id=2d24208b-94ce-4364-86a4-1758c24e9047,network=Network(69963625-cff2-4260-908f-82359d8b2f77),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d24208b-94') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG os_vif [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:22:c7:b3,bridge_name='br-int',has_traffic_filtering=True,id=2d24208b-94ce-4364-86a4-1758c24e9047,network=Network(69963625-cff2-4260-908f-82359d8b2f77),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d24208b-94') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json" returned: 0 in 0.225s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2d24208b-94, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2d24208b-94, col_values=(('external_ids', {'iface-id': '2d24208b-94ce-4364-86a4-1758c24e9047', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:22:c7:b3', 'vm-uuid': 'e5344b0d-2d73-40c7-8ac3-535396e985a4'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Cannot resize image /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lazy-loading 'migration_context' on Instance uuid 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:17:08 user nova-compute[71317]: INFO os_vif [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:22:c7:b3,bridge_name='br-int',has_traffic_filtering=True,id=2d24208b-94ce-4364-86a4-1758c24e9047,network=Network(69963625-cff2-4260-908f-82359d8b2f77),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d24208b-94')
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Ensure instance console log exists: /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.network.neutron [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Successfully created port: 60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] No BDM found with device name sda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] No BDM found with device name sdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:17:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] No VIF found with MAC fa:16:3e:22:c7:b3, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:17:08 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Using config drive
Apr 14 04:17:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Creating config drive at /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk.config
Apr 14 04:17:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpgqrs8e70 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:09 user nova-compute[71317]: DEBUG nova.network.neutron [req-a9c3c8a1-195b-48a2-b871-3fa09d0a5629 req-49e00a46-01eb-4850-a131-78d2ea502762 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Updated VIF entry in instance network info cache for port 2d24208b-94ce-4364-86a4-1758c24e9047. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:17:09 user nova-compute[71317]: DEBUG nova.network.neutron [req-a9c3c8a1-195b-48a2-b871-3fa09d0a5629 req-49e00a46-01eb-4850-a131-78d2ea502762 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Updating instance_info_cache with network_info: [{"id": "2d24208b-94ce-4364-86a4-1758c24e9047", "address": "fa:16:3e:22:c7:b3", "network": {"id": "69963625-cff2-4260-908f-82359d8b2f77", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-558617934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e0e636ba0ee9486f9f48905d886ab547", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d24208b-94", "ovs_interfaceid": "2d24208b-94ce-4364-86a4-1758c24e9047", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpgqrs8e70" returned: 0 in 0.074s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a9c3c8a1-195b-48a2-b871-3fa09d0a5629 req-49e00a46-01eb-4850-a131-78d2ea502762 service nova] Releasing lock "refresh_cache-e5344b0d-2d73-40c7-8ac3-535396e985a4" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.network.neutron [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Successfully updated port: 60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquired lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.network.neutron [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-7ad9acef-447d-44e6-8af8-abea81977a44 req-8d5417dd-4d0e-4a3a-bb06-5ae67b543684 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-changed-60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-7ad9acef-447d-44e6-8af8-abea81977a44 req-8d5417dd-4d0e-4a3a-bb06-5ae67b543684 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Refreshing instance network info cache due to event network-changed-60579bc4-de13-48d7-b42d-2b9fc5597e45. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7ad9acef-447d-44e6-8af8-abea81977a44 req-8d5417dd-4d0e-4a3a-bb06-5ae67b543684 service nova] Acquiring lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.network.neutron [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.network.neutron [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Updating instance_info_cache with network_info: [{"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Releasing lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Instance network_info: |[{"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7ad9acef-447d-44e6-8af8-abea81977a44 req-8d5417dd-4d0e-4a3a-bb06-5ae67b543684 service nova] Acquired lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.network.neutron [req-7ad9acef-447d-44e6-8af8-abea81977a44 req-8d5417dd-4d0e-4a3a-bb06-5ae67b543684 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Refreshing network info cache for port 60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Start _get_guest_xml network_info=[{"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:17:10 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:17:10 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-420804238',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-420804238',id=10,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBvOf9hss+Yqylm6PnX6la1ymm/55r9GRsHJ0NsNe4bgFVG/3wZy3kBc49f/xLbrJ51dzLLHr/o6IQVXcYhlDKHgp4DCKWnVv/anFtE+/Na/vdZgAYvQkLcZYHFyD4dFRQ==',key_name='tempest-keypair-330344245',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='01cefa66350a4432ad71204b39e953be',ramdisk_id='',reservation_id='r-p91nk3xp',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedAttachmentsTest-926384577',owner_user_name='tempest-TaggedAttachmentsTest-926384577-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:08Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='0aeeeae7cbc340daa8cd81c1418794e0',uuid=8ed1dea8-e48a-49f7-924e-82cf639a1677,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converting VIF {"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:58:93:64,bridge_name='br-int',has_traffic_filtering=True,id=60579bc4-de13-48d7-b42d-2b9fc5597e45,network=Network(c9babf6f-d879-4cff-b628-f100d9293bff),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap60579bc4-de') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lazy-loading 'pci_devices' on Instance uuid 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:17:10 user nova-compute[71317]:   <uuid>8ed1dea8-e48a-49f7-924e-82cf639a1677</uuid>
Apr 14 04:17:10 user nova-compute[71317]:   <name>instance-0000000a</name>
Apr 14 04:17:10 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:17:10 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:17:10 user nova-compute[71317]:   <metadata>
Apr 14 04:17:10 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:17:10 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:17:10 user nova-compute[71317]:       <nova:name>tempest-device-tagging-server-420804238</nova:name>
Apr 14 04:17:10 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:17:10</nova:creationTime>
Apr 14 04:17:10 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:17:10 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:17:10 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:17:10 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:17:10 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:17:10 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:17:10 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:17:10 user nova-compute[71317]:       <nova:owner>
Apr 14 04:17:10 user nova-compute[71317]:         <nova:user uuid="0aeeeae7cbc340daa8cd81c1418794e0">tempest-TaggedAttachmentsTest-926384577-project-member</nova:user>
Apr 14 04:17:10 user nova-compute[71317]:         <nova:project uuid="01cefa66350a4432ad71204b39e953be">tempest-TaggedAttachmentsTest-926384577</nova:project>
Apr 14 04:17:10 user nova-compute[71317]:       </nova:owner>
Apr 14 04:17:10 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:17:10 user nova-compute[71317]:       <nova:ports>
Apr 14 04:17:10 user nova-compute[71317]:         <nova:port uuid="60579bc4-de13-48d7-b42d-2b9fc5597e45">
Apr 14 04:17:10 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 04:17:10 user nova-compute[71317]:         </nova:port>
Apr 14 04:17:10 user nova-compute[71317]:       </nova:ports>
Apr 14 04:17:10 user nova-compute[71317]:     </nova:instance>
Apr 14 04:17:10 user nova-compute[71317]:   </metadata>
Apr 14 04:17:10 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:17:10 user nova-compute[71317]:     <system>
Apr 14 04:17:10 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:17:10 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:17:10 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:17:10 user nova-compute[71317]:       <entry name="serial">8ed1dea8-e48a-49f7-924e-82cf639a1677</entry>
Apr 14 04:17:10 user nova-compute[71317]:       <entry name="uuid">8ed1dea8-e48a-49f7-924e-82cf639a1677</entry>
Apr 14 04:17:10 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:17:10 user nova-compute[71317]:     </system>
Apr 14 04:17:10 user nova-compute[71317]:   </sysinfo>
Apr 14 04:17:10 user nova-compute[71317]:   <os>
Apr 14 04:17:10 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:17:10 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:17:10 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:17:10 user nova-compute[71317]:   </os>
Apr 14 04:17:10 user nova-compute[71317]:   <features>
Apr 14 04:17:10 user nova-compute[71317]:     <acpi/>
Apr 14 04:17:10 user nova-compute[71317]:     <apic/>
Apr 14 04:17:10 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:17:10 user nova-compute[71317]:   </features>
Apr 14 04:17:10 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:17:10 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:17:10 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:17:10 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:17:10 user nova-compute[71317]:   </clock>
Apr 14 04:17:10 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:17:10 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:17:10 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:17:10 user nova-compute[71317]:   </cpu>
Apr 14 04:17:10 user nova-compute[71317]:   <devices>
Apr 14 04:17:10 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:17:10 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:17:10 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk"/>
Apr 14 04:17:10 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:17:10 user nova-compute[71317]:     </disk>
Apr 14 04:17:10 user nova-compute[71317]:     <disk type="file" device="cdrom">
Apr 14 04:17:10 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none"/>
Apr 14 04:17:10 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk.config"/>
Apr 14 04:17:10 user nova-compute[71317]:       <target dev="hda" bus="ide"/>
Apr 14 04:17:10 user nova-compute[71317]:     </disk>
Apr 14 04:17:10 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:17:10 user nova-compute[71317]:       <mac address="fa:16:3e:58:93:64"/>
Apr 14 04:17:10 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:17:10 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:17:10 user nova-compute[71317]:       <target dev="tap60579bc4-de"/>
Apr 14 04:17:10 user nova-compute[71317]:     </interface>
Apr 14 04:17:10 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:17:10 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/console.log" append="off"/>
Apr 14 04:17:10 user nova-compute[71317]:     </serial>
Apr 14 04:17:10 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:17:10 user nova-compute[71317]:     <video>
Apr 14 04:17:10 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:17:10 user nova-compute[71317]:     </video>
Apr 14 04:17:10 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:17:10 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:17:10 user nova-compute[71317]:     </rng>
Apr 14 04:17:10 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:17:10 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:17:10 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:17:10 user nova-compute[71317]:     </memballoon>
Apr 14 04:17:10 user nova-compute[71317]:   </devices>
Apr 14 04:17:10 user nova-compute[71317]: </domain>
Apr 14 04:17:10 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-420804238',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-420804238',id=10,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBvOf9hss+Yqylm6PnX6la1ymm/55r9GRsHJ0NsNe4bgFVG/3wZy3kBc49f/xLbrJ51dzLLHr/o6IQVXcYhlDKHgp4DCKWnVv/anFtE+/Na/vdZgAYvQkLcZYHFyD4dFRQ==',key_name='tempest-keypair-330344245',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='01cefa66350a4432ad71204b39e953be',ramdisk_id='',reservation_id='r-p91nk3xp',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedAttachmentsTest-926384577',owner_user_name='tempest-TaggedAttachmentsTest-926384577-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:08Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='0aeeeae7cbc340daa8cd81c1418794e0',uuid=8ed1dea8-e48a-49f7-924e-82cf639a1677,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converting VIF {"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:58:93:64,bridge_name='br-int',has_traffic_filtering=True,id=60579bc4-de13-48d7-b42d-2b9fc5597e45,network=Network(c9babf6f-d879-4cff-b628-f100d9293bff),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap60579bc4-de') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG os_vif [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:58:93:64,bridge_name='br-int',has_traffic_filtering=True,id=60579bc4-de13-48d7-b42d-2b9fc5597e45,network=Network(c9babf6f-d879-4cff-b628-f100d9293bff),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap60579bc4-de') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap60579bc4-de, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap60579bc4-de, col_values=(('external_ids', {'iface-id': '60579bc4-de13-48d7-b42d-2b9fc5597e45', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:58:93:64', 'vm-uuid': '8ed1dea8-e48a-49f7-924e-82cf639a1677'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:10 user nova-compute[71317]: INFO os_vif [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:58:93:64,bridge_name='br-int',has_traffic_filtering=True,id=60579bc4-de13-48d7-b42d-2b9fc5597e45,network=Network(c9babf6f-d879-4cff-b628-f100d9293bff),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap60579bc4-de')
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] No BDM found with device name hda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] No VIF found with MAC fa:16:3e:58:93:64, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:17:10 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Using config drive
Apr 14 04:17:10 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Creating config drive at /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk.config
Apr 14 04:17:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpbcbpnflo {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpbcbpnflo" returned: 0 in 0.098s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:11 user nova-compute[71317]: DEBUG nova.network.neutron [req-7ad9acef-447d-44e6-8af8-abea81977a44 req-8d5417dd-4d0e-4a3a-bb06-5ae67b543684 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Updated VIF entry in instance network info cache for port 60579bc4-de13-48d7-b42d-2b9fc5597e45. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:17:11 user nova-compute[71317]: DEBUG nova.network.neutron [req-7ad9acef-447d-44e6-8af8-abea81977a44 req-8d5417dd-4d0e-4a3a-bb06-5ae67b543684 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Updating instance_info_cache with network_info: [{"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7ad9acef-447d-44e6-8af8-abea81977a44 req-8d5417dd-4d0e-4a3a-bb06-5ae67b543684 service nova] Releasing lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-29e54fbe-56a3-4de8-8d53-1278002dde35 req-fd92b667-fc20-4bdb-aa66-53762509bc71 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Received event network-vif-plugged-2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-29e54fbe-56a3-4de8-8d53-1278002dde35 req-fd92b667-fc20-4bdb-aa66-53762509bc71 service nova] Acquiring lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-29e54fbe-56a3-4de8-8d53-1278002dde35 req-fd92b667-fc20-4bdb-aa66-53762509bc71 service nova] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-29e54fbe-56a3-4de8-8d53-1278002dde35 req-fd92b667-fc20-4bdb-aa66-53762509bc71 service nova] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-29e54fbe-56a3-4de8-8d53-1278002dde35 req-fd92b667-fc20-4bdb-aa66-53762509bc71 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] No waiting events found dispatching network-vif-plugged-2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:17:11 user nova-compute[71317]: WARNING nova.compute.manager [req-29e54fbe-56a3-4de8-8d53-1278002dde35 req-fd92b667-fc20-4bdb-aa66-53762509bc71 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Received unexpected event network-vif-plugged-2d24208b-94ce-4364-86a4-1758c24e9047 for instance with vm_state building and task_state spawning.
Apr 14 04:17:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:17:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:17:12 user nova-compute[71317]: INFO nova.compute.claims [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Claim successful on node user
Apr 14 04:17:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e891da9-cd62-41cc-b4a9-98b35d2b5620 req-760dc850-7864-46e8-a228-2174b5dea0f0 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-vif-plugged-60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e891da9-cd62-41cc-b4a9-98b35d2b5620 req-760dc850-7864-46e8-a228-2174b5dea0f0 service nova] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e891da9-cd62-41cc-b4a9-98b35d2b5620 req-760dc850-7864-46e8-a228-2174b5dea0f0 service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e891da9-cd62-41cc-b4a9-98b35d2b5620 req-760dc850-7864-46e8-a228-2174b5dea0f0 service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e891da9-cd62-41cc-b4a9-98b35d2b5620 req-760dc850-7864-46e8-a228-2174b5dea0f0 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] No waiting events found dispatching network-vif-plugged-60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:17:13 user nova-compute[71317]: WARNING nova.compute.manager [req-5e891da9-cd62-41cc-b4a9-98b35d2b5620 req-760dc850-7864-46e8-a228-2174b5dea0f0 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received unexpected event network-vif-plugged-60579bc4-de13-48d7-b42d-2b9fc5597e45 for instance with vm_state building and task_state spawning.
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.848s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.network.neutron [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:17:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-39fb2c73-d4d5-4fc7-9e35-4bc1cfe0d3bc req-c841a8a3-3851-44ab-a90c-3391af63cb76 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Received event network-vif-plugged-2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-39fb2c73-d4d5-4fc7-9e35-4bc1cfe0d3bc req-c841a8a3-3851-44ab-a90c-3391af63cb76 service nova] Acquiring lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-39fb2c73-d4d5-4fc7-9e35-4bc1cfe0d3bc req-c841a8a3-3851-44ab-a90c-3391af63cb76 service nova] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-39fb2c73-d4d5-4fc7-9e35-4bc1cfe0d3bc req-c841a8a3-3851-44ab-a90c-3391af63cb76 service nova] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-39fb2c73-d4d5-4fc7-9e35-4bc1cfe0d3bc req-c841a8a3-3851-44ab-a90c-3391af63cb76 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] No waiting events found dispatching network-vif-plugged-2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:17:13 user nova-compute[71317]: WARNING nova.compute.manager [req-39fb2c73-d4d5-4fc7-9e35-4bc1cfe0d3bc req-c841a8a3-3851-44ab-a90c-3391af63cb76 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Received unexpected event network-vif-plugged-2d24208b-94ce-4364-86a4-1758c24e9047 for instance with vm_state building and task_state spawning.
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:17:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Creating image(s)
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "/opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "/opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "/opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG nova.policy [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '1da030e7c9e44f9b82f3286378a2ede6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ffadee5eeb7f4b42ae93e2e2578ef897', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.166s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk 1073741824" returned: 0 in 0.066s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.243s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.160s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Checking if we can resize image /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Cannot resize image /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG nova.objects.instance [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lazy-loading 'migration_context' on Instance uuid b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Ensure instance console log exists: /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:14 user nova-compute[71317]: DEBUG nova.network.neutron [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Successfully created port: 3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445835.0010123, e5344b0d-2d73-40c7-8ac3-535396e985a4 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] VM Resumed (Lifecycle Event)
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Instance spawned successfully.
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445835.0149076, e5344b0d-2d73-40c7-8ac3-535396e985a4 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] VM Started (Lifecycle Event)
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.compute.manager [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Took 10.34 seconds to spawn the instance on the hypervisor.
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.compute.manager [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Took 11.85 seconds to build instance.
Apr 14 04:17:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e3076907-3a1c-41e0-ae3e-83882698d70e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 12.078s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-a913dbf1-3b68-4b45-b31d-630894ff0441 req-3e4e493a-27ab-4d1b-ae71-1a384d70cf1a service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-vif-plugged-60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a913dbf1-3b68-4b45-b31d-630894ff0441 req-3e4e493a-27ab-4d1b-ae71-1a384d70cf1a service nova] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a913dbf1-3b68-4b45-b31d-630894ff0441 req-3e4e493a-27ab-4d1b-ae71-1a384d70cf1a service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a913dbf1-3b68-4b45-b31d-630894ff0441 req-3e4e493a-27ab-4d1b-ae71-1a384d70cf1a service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-a913dbf1-3b68-4b45-b31d-630894ff0441 req-3e4e493a-27ab-4d1b-ae71-1a384d70cf1a service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] No waiting events found dispatching network-vif-plugged-60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:17:15 user nova-compute[71317]: WARNING nova.compute.manager [req-a913dbf1-3b68-4b45-b31d-630894ff0441 req-3e4e493a-27ab-4d1b-ae71-1a384d70cf1a service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received unexpected event network-vif-plugged-60579bc4-de13-48d7-b42d-2b9fc5597e45 for instance with vm_state building and task_state spawning.
Apr 14 04:17:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445835.59523, 8ed1dea8-e48a-49f7-924e-82cf639a1677 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] VM Resumed (Lifecycle Event)
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Instance spawned successfully.
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445835.596132, 8ed1dea8-e48a-49f7-924e-82cf639a1677 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] VM Started (Lifecycle Event)
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.compute.manager [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Took 8.03 seconds to spawn the instance on the hypervisor.
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.network.neutron [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Successfully updated port: 3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "refresh_cache-b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquired lock "refresh_cache-b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.network.neutron [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:17:15 user nova-compute[71317]: INFO nova.compute.manager [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Took 9.61 seconds to build instance.
Apr 14 04:17:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cf0cda3-a7cf-4354-9e51-6a8ecc7d721d tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.940s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-cf61eaee-4c5e-4241-b47a-25c28c9a3df2 req-f8b6e73d-d591-401d-b3d6-2eb72b4f958d service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Received event network-changed-3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-cf61eaee-4c5e-4241-b47a-25c28c9a3df2 req-f8b6e73d-d591-401d-b3d6-2eb72b4f958d service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Refreshing instance network info cache due to event network-changed-3f178241-75e2-4e0a-8d92-e647d05ed85e. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:17:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cf61eaee-4c5e-4241-b47a-25c28c9a3df2 req-f8b6e73d-d591-401d-b3d6-2eb72b4f958d service nova] Acquiring lock "refresh_cache-b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.network.neutron [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.network.neutron [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Updating instance_info_cache with network_info: [{"id": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "address": "fa:16:3e:cd:d0:ea", "network": {"id": "43bde039-802b-47d6-9194-3fcc8d89149f", "bridge": "br-int", "label": "tempest-TestVolumeSwap-57267579-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ffadee5eeb7f4b42ae93e2e2578ef897", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3f178241-75", "ovs_interfaceid": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Releasing lock "refresh_cache-b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Instance network_info: |[{"id": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "address": "fa:16:3e:cd:d0:ea", "network": {"id": "43bde039-802b-47d6-9194-3fcc8d89149f", "bridge": "br-int", "label": "tempest-TestVolumeSwap-57267579-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ffadee5eeb7f4b42ae93e2e2578ef897", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3f178241-75", "ovs_interfaceid": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cf61eaee-4c5e-4241-b47a-25c28c9a3df2 req-f8b6e73d-d591-401d-b3d6-2eb72b4f958d service nova] Acquired lock "refresh_cache-b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.network.neutron [req-cf61eaee-4c5e-4241-b47a-25c28c9a3df2 req-f8b6e73d-d591-401d-b3d6-2eb72b4f958d service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Refreshing network info cache for port 3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Start _get_guest_xml network_info=[{"id": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "address": "fa:16:3e:cd:d0:ea", "network": {"id": "43bde039-802b-47d6-9194-3fcc8d89149f", "bridge": "br-int", "label": "tempest-TestVolumeSwap-57267579-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ffadee5eeb7f4b42ae93e2e2578ef897", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3f178241-75", "ovs_interfaceid": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:17:16 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:17:16 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:17:11Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-1758802671',display_name='tempest-TestVolumeSwap-server-1758802671',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumeswap-server-1758802671',id=11,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFbgdbG+/8WAlmHZQQgAOqS/UONGNW8t82kEvX2AWKVyIPVWzo6LlVeaPM998LCX+uM50LF1wQnPotgqjtWl9nx8bNCNwqqSA4Fxg3g55HZulk97rrGLje8WCx8z8LeJ+Q==',key_name='tempest-keypair-1360289330',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ffadee5eeb7f4b42ae93e2e2578ef897',ramdisk_id='',reservation_id='r-85pcdwwm',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeSwap-926061730',owner_user_name='tempest-TestVolumeSwap-926061730-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='1da030e7c9e44f9b82f3286378a2ede6',uuid=b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "address": "fa:16:3e:cd:d0:ea", "network": {"id": "43bde039-802b-47d6-9194-3fcc8d89149f", "bridge": "br-int", "label": "tempest-TestVolumeSwap-57267579-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ffadee5eeb7f4b42ae93e2e2578ef897", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3f178241-75", "ovs_interfaceid": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Converting VIF {"id": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "address": "fa:16:3e:cd:d0:ea", "network": {"id": "43bde039-802b-47d6-9194-3fcc8d89149f", "bridge": "br-int", "label": "tempest-TestVolumeSwap-57267579-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ffadee5eeb7f4b42ae93e2e2578ef897", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3f178241-75", "ovs_interfaceid": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:cd:d0:ea,bridge_name='br-int',has_traffic_filtering=True,id=3f178241-75e2-4e0a-8d92-e647d05ed85e,network=Network(43bde039-802b-47d6-9194-3fcc8d89149f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3f178241-75') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.objects.instance [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lazy-loading 'pci_devices' on Instance uuid b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:17:16 user nova-compute[71317]:   <uuid>b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7</uuid>
Apr 14 04:17:16 user nova-compute[71317]:   <name>instance-0000000b</name>
Apr 14 04:17:16 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:17:16 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:17:16 user nova-compute[71317]:   <metadata>
Apr 14 04:17:16 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:17:16 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:17:16 user nova-compute[71317]:       <nova:name>tempest-TestVolumeSwap-server-1758802671</nova:name>
Apr 14 04:17:16 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:17:16</nova:creationTime>
Apr 14 04:17:16 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:17:16 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:17:16 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:17:16 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:17:16 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:17:16 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:17:16 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:17:16 user nova-compute[71317]:       <nova:owner>
Apr 14 04:17:16 user nova-compute[71317]:         <nova:user uuid="1da030e7c9e44f9b82f3286378a2ede6">tempest-TestVolumeSwap-926061730-project-member</nova:user>
Apr 14 04:17:16 user nova-compute[71317]:         <nova:project uuid="ffadee5eeb7f4b42ae93e2e2578ef897">tempest-TestVolumeSwap-926061730</nova:project>
Apr 14 04:17:16 user nova-compute[71317]:       </nova:owner>
Apr 14 04:17:16 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:17:16 user nova-compute[71317]:       <nova:ports>
Apr 14 04:17:16 user nova-compute[71317]:         <nova:port uuid="3f178241-75e2-4e0a-8d92-e647d05ed85e">
Apr 14 04:17:16 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 04:17:16 user nova-compute[71317]:         </nova:port>
Apr 14 04:17:16 user nova-compute[71317]:       </nova:ports>
Apr 14 04:17:16 user nova-compute[71317]:     </nova:instance>
Apr 14 04:17:16 user nova-compute[71317]:   </metadata>
Apr 14 04:17:16 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:17:16 user nova-compute[71317]:     <system>
Apr 14 04:17:16 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:17:16 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:17:16 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:17:16 user nova-compute[71317]:       <entry name="serial">b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7</entry>
Apr 14 04:17:16 user nova-compute[71317]:       <entry name="uuid">b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7</entry>
Apr 14 04:17:16 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:17:16 user nova-compute[71317]:     </system>
Apr 14 04:17:16 user nova-compute[71317]:   </sysinfo>
Apr 14 04:17:16 user nova-compute[71317]:   <os>
Apr 14 04:17:16 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:17:16 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:17:16 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:17:16 user nova-compute[71317]:   </os>
Apr 14 04:17:16 user nova-compute[71317]:   <features>
Apr 14 04:17:16 user nova-compute[71317]:     <acpi/>
Apr 14 04:17:16 user nova-compute[71317]:     <apic/>
Apr 14 04:17:16 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:17:16 user nova-compute[71317]:   </features>
Apr 14 04:17:16 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:17:16 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:17:16 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:17:16 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:17:16 user nova-compute[71317]:   </clock>
Apr 14 04:17:16 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:17:16 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:17:16 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:17:16 user nova-compute[71317]:   </cpu>
Apr 14 04:17:16 user nova-compute[71317]:   <devices>
Apr 14 04:17:16 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:17:16 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:17:16 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk"/>
Apr 14 04:17:16 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:17:16 user nova-compute[71317]:     </disk>
Apr 14 04:17:16 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:17:16 user nova-compute[71317]:       <mac address="fa:16:3e:cd:d0:ea"/>
Apr 14 04:17:16 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:17:16 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:17:16 user nova-compute[71317]:       <target dev="tap3f178241-75"/>
Apr 14 04:17:16 user nova-compute[71317]:     </interface>
Apr 14 04:17:16 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:17:16 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/console.log" append="off"/>
Apr 14 04:17:16 user nova-compute[71317]:     </serial>
Apr 14 04:17:16 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:17:16 user nova-compute[71317]:     <video>
Apr 14 04:17:16 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:17:16 user nova-compute[71317]:     </video>
Apr 14 04:17:16 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:17:16 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:17:16 user nova-compute[71317]:     </rng>
Apr 14 04:17:16 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:17:16 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:17:16 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:17:16 user nova-compute[71317]:     </memballoon>
Apr 14 04:17:16 user nova-compute[71317]:   </devices>
Apr 14 04:17:16 user nova-compute[71317]: </domain>
Apr 14 04:17:16 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:17:11Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-1758802671',display_name='tempest-TestVolumeSwap-server-1758802671',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumeswap-server-1758802671',id=11,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFbgdbG+/8WAlmHZQQgAOqS/UONGNW8t82kEvX2AWKVyIPVWzo6LlVeaPM998LCX+uM50LF1wQnPotgqjtWl9nx8bNCNwqqSA4Fxg3g55HZulk97rrGLje8WCx8z8LeJ+Q==',key_name='tempest-keypair-1360289330',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ffadee5eeb7f4b42ae93e2e2578ef897',ramdisk_id='',reservation_id='r-85pcdwwm',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeSwap-926061730',owner_user_name='tempest-TestVolumeSwap-926061730-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='1da030e7c9e44f9b82f3286378a2ede6',uuid=b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "address": "fa:16:3e:cd:d0:ea", "network": {"id": "43bde039-802b-47d6-9194-3fcc8d89149f", "bridge": "br-int", "label": "tempest-TestVolumeSwap-57267579-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ffadee5eeb7f4b42ae93e2e2578ef897", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3f178241-75", "ovs_interfaceid": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Converting VIF {"id": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "address": "fa:16:3e:cd:d0:ea", "network": {"id": "43bde039-802b-47d6-9194-3fcc8d89149f", "bridge": "br-int", "label": "tempest-TestVolumeSwap-57267579-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ffadee5eeb7f4b42ae93e2e2578ef897", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3f178241-75", "ovs_interfaceid": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:cd:d0:ea,bridge_name='br-int',has_traffic_filtering=True,id=3f178241-75e2-4e0a-8d92-e647d05ed85e,network=Network(43bde039-802b-47d6-9194-3fcc8d89149f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3f178241-75') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG os_vif [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:cd:d0:ea,bridge_name='br-int',has_traffic_filtering=True,id=3f178241-75e2-4e0a-8d92-e647d05ed85e,network=Network(43bde039-802b-47d6-9194-3fcc8d89149f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3f178241-75') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3f178241-75, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap3f178241-75, col_values=(('external_ids', {'iface-id': '3f178241-75e2-4e0a-8d92-e647d05ed85e', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:cd:d0:ea', 'vm-uuid': 'b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:17:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:16 user nova-compute[71317]: INFO os_vif [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:cd:d0:ea,bridge_name='br-int',has_traffic_filtering=True,id=3f178241-75e2-4e0a-8d92-e647d05ed85e,network=Network(43bde039-802b-47d6-9194-3fcc8d89149f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3f178241-75')
Apr 14 04:17:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:17:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] No VIF found with MAC fa:16:3e:cd:d0:ea, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:17:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:17 user nova-compute[71317]: DEBUG nova.network.neutron [req-cf61eaee-4c5e-4241-b47a-25c28c9a3df2 req-f8b6e73d-d591-401d-b3d6-2eb72b4f958d service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Updated VIF entry in instance network info cache for port 3f178241-75e2-4e0a-8d92-e647d05ed85e. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:17:17 user nova-compute[71317]: DEBUG nova.network.neutron [req-cf61eaee-4c5e-4241-b47a-25c28c9a3df2 req-f8b6e73d-d591-401d-b3d6-2eb72b4f958d service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Updating instance_info_cache with network_info: [{"id": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "address": "fa:16:3e:cd:d0:ea", "network": {"id": "43bde039-802b-47d6-9194-3fcc8d89149f", "bridge": "br-int", "label": "tempest-TestVolumeSwap-57267579-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ffadee5eeb7f4b42ae93e2e2578ef897", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3f178241-75", "ovs_interfaceid": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cf61eaee-4c5e-4241-b47a-25c28c9a3df2 req-f8b6e73d-d591-401d-b3d6-2eb72b4f958d service nova] Releasing lock "refresh_cache-b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:17:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 04:17:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] There are 0 instances to clean {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 04:17:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:17:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances with incomplete migration  {{(pid=71317) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 04:17:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:17:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-9548a05c-8a2b-4256-94ea-f86fbdb85754 req-d9b7aa95-bf31-407f-8e38-1e9e1a317990 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Received event network-vif-plugged-3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9548a05c-8a2b-4256-94ea-f86fbdb85754 req-d9b7aa95-bf31-407f-8e38-1e9e1a317990 service nova] Acquiring lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9548a05c-8a2b-4256-94ea-f86fbdb85754 req-d9b7aa95-bf31-407f-8e38-1e9e1a317990 service nova] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9548a05c-8a2b-4256-94ea-f86fbdb85754 req-d9b7aa95-bf31-407f-8e38-1e9e1a317990 service nova] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-9548a05c-8a2b-4256-94ea-f86fbdb85754 req-d9b7aa95-bf31-407f-8e38-1e9e1a317990 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] No waiting events found dispatching network-vif-plugged-3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:17:19 user nova-compute[71317]: WARNING nova.compute.manager [req-9548a05c-8a2b-4256-94ea-f86fbdb85754 req-d9b7aa95-bf31-407f-8e38-1e9e1a317990 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Received unexpected event network-vif-plugged-3f178241-75e2-4e0a-8d92-e647d05ed85e for instance with vm_state building and task_state spawning.
Apr 14 04:17:19 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:17:19 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:17:19 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:17:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:20 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-7ac42c71-1388-46ed-bdd8-28ff0d85a5f2 req-7b4ca8df-2d2a-4c99-9405-9afbcef6f960 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Received event network-vif-plugged-3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7ac42c71-1388-46ed-bdd8-28ff0d85a5f2 req-7b4ca8df-2d2a-4c99-9405-9afbcef6f960 service nova] Acquiring lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7ac42c71-1388-46ed-bdd8-28ff0d85a5f2 req-7b4ca8df-2d2a-4c99-9405-9afbcef6f960 service nova] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7ac42c71-1388-46ed-bdd8-28ff0d85a5f2 req-7b4ca8df-2d2a-4c99-9405-9afbcef6f960 service nova] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-7ac42c71-1388-46ed-bdd8-28ff0d85a5f2 req-7b4ca8df-2d2a-4c99-9405-9afbcef6f960 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] No waiting events found dispatching network-vif-plugged-3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:17:21 user nova-compute[71317]: WARNING nova.compute.manager [req-7ac42c71-1388-46ed-bdd8-28ff0d85a5f2 req-7b4ca8df-2d2a-4c99-9405-9afbcef6f960 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Received unexpected event network-vif-plugged-3f178241-75e2-4e0a-8d92-e647d05ed85e for instance with vm_state building and task_state spawning.
Apr 14 04:17:21 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Skipping network cache update for instance because it is Building. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9805}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-0ff97860-2441-437c-b118-6089c3cc4160" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-0ff97860-2441-437c-b118-6089c3cc4160" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lazy-loading 'info_cache' on Instance uuid 0ff97860-2441-437c-b118-6089c3cc4160 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445841.4997537, b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] VM Resumed (Lifecycle Event)
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:17:21 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Instance spawned successfully.
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:17:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445841.500782, b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:17:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] VM Started (Lifecycle Event)
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:17:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:17:21 user nova-compute[71317]: INFO nova.compute.manager [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Took 7.95 seconds to spawn the instance on the hypervisor.
Apr 14 04:17:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:17:21 user nova-compute[71317]: INFO nova.compute.manager [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Took 9.36 seconds to build instance.
Apr 14 04:17:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c0535bc-79b9-452e-a3e5-2ef7cc727327 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.487s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Updating instance_info_cache with network_info: [{"id": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "address": "fa:16:3e:41:ed:a5", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22212b8b-66", "ovs_interfaceid": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-0ff97860-2441-437c-b118-6089c3cc4160" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.217s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json" returned: 0 in 0.211s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json" returned: 0 in 0.196s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk --force-share --output=json" returned: 0 in 0.253s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.191s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk --force-share --output=json" returned: 0 in 0.186s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:26 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:17:26 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:17:26 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7360MB free_disk=26.656696319580078GB free_vcpus=1 pci_devices=[{"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:17:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 0ff97860-2441-437c-b118-6089c3cc4160 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance da3b4651-aaa0-430d-85b8-c15de386d08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 9b532a44-20b7-4590-babe-684c49598a62 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance e37e27b4-0813-4df9-ac66-1516334c12cb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance e5344b0d-2d73-40c7-8ac3-535396e985a4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1920MB phys_disk=40GB used_disk=11GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing inventories for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating ProviderTree inventory for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating inventory in ProviderTree for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing aggregate associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, aggregates: None {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing trait associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, traits: COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSE41,HW_CPU_X86_SSE42,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_TRUSTED_CERTS,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_ACCELERATORS,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NET_VIF_MODEL_E1000,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_NODE,HW_CPU_X86_SSE2,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,HW_CPU_X86_SSSE3,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_STORAGE_BUS_USB,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_NE2K_PCI {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:17:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.892s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:17:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:17:42 user nova-compute[71317]: INFO nova.compute.claims [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Claim successful on node user
Apr 14 04:17:42 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.444s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:17:42 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:17:42 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:17:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:17:43 user nova-compute[71317]: INFO nova.virt.block_device [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Booting with volume 31382528-1966-4f19-8a4a-94983a6314c3 at /dev/vda
Apr 14 04:17:43 user nova-compute[71317]: DEBUG nova.policy [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b848680106b74275a947f316ce6311fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '225e61ca983d4f27adc61ad33c81fce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:17:43 user nova-compute[71317]: DEBUG os_brick.utils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:17:43 user nova-compute[71317]: INFO oslo.privsep.daemon [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpgvuke2fr/privsep.sock']
Apr 14 04:17:43 user sudo[81934]:    stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmpgvuke2fr/privsep.sock
Apr 14 04:17:43 user sudo[81934]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Apr 14 04:17:43 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Successfully created port: eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:17:44 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Successfully created port: 11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:17:44 user sudo[81934]: pam_unix(sudo:session): session closed for user root
Apr 14 04:17:44 user nova-compute[71317]: INFO oslo.privsep.daemon [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Spawned new privsep daemon via rootwrap
Apr 14 04:17:44 user nova-compute[71317]: INFO oslo.privsep.daemon [-] privsep daemon starting
Apr 14 04:17:44 user nova-compute[71317]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Apr 14 04:17:44 user nova-compute[71317]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none
Apr 14 04:17:44 user nova-compute[71317]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 81937
Apr 14 04:17:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[42c3ae6c-deff-490a-90d8-1b56368e0260]: (2,) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6725d9d6-0b4c-4ddb-91c3-bb466dcd0681]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:45 user nova-compute[71317]: WARNING os_brick.initiator.connectors.nvmeof [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Could not find nvme_core/parameters/multipath: FileNotFoundError: [Errno 2] No such file or directory: '/sys/module/nvme_core/parameters/multipath'
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[911120a1-a528-4481-97a2-6f3f1d1b23c1]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Successfully created port: 8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[514d7c38-f058-4275-ac10-1cd1c44f0f6f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4796b8c9-cc59-4414-8660-1375b62a1d2a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:17:45 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:17:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a56fe4ca-ab5c-4c40-9c09-bb453d522901]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG os_brick.utils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] <== get_connector_properties: return (1967ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating existing volume attachment record: e342f29f-eaa0-4d55-bbe5-e2dd4fd88e97 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:17:45 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Successfully created port: 07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:17:46 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Successfully created port: 512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:17:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:46 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Successfully updated port: eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:17:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-653ae4f7-4300-4540-93b5-85b7da30fb72 req-d4a1d0c7-bbba-4a4f-8e66-758a487a9cef service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-changed-eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-653ae4f7-4300-4540-93b5-85b7da30fb72 req-d4a1d0c7-bbba-4a4f-8e66-758a487a9cef service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing instance network info cache due to event network-changed-eea29fce-f6e3-4715-9a46-2a638462c6f8. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:17:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-653ae4f7-4300-4540-93b5-85b7da30fb72 req-d4a1d0c7-bbba-4a4f-8e66-758a487a9cef service nova] Acquiring lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-653ae4f7-4300-4540-93b5-85b7da30fb72 req-d4a1d0c7-bbba-4a4f-8e66-758a487a9cef service nova] Acquired lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:47 user nova-compute[71317]: DEBUG nova.network.neutron [req-653ae4f7-4300-4540-93b5-85b7da30fb72 req-d4a1d0c7-bbba-4a4f-8e66-758a487a9cef service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing network info cache for port eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:17:47 user nova-compute[71317]: DEBUG nova.network.neutron [req-653ae4f7-4300-4540-93b5-85b7da30fb72 req-d4a1d0c7-bbba-4a4f-8e66-758a487a9cef service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:17:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:47 user nova-compute[71317]: DEBUG nova.network.neutron [req-653ae4f7-4300-4540-93b5-85b7da30fb72 req-d4a1d0c7-bbba-4a4f-8e66-758a487a9cef service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-653ae4f7-4300-4540-93b5-85b7da30fb72 req-d4a1d0c7-bbba-4a4f-8e66-758a487a9cef service nova] Releasing lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:48 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Successfully updated port: 8c995dcb-b43b-4bdd-b631-e3b6104d49d0 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-870a5631-2ce1-425e-92ca-d781f868f225 req-4ce0e604-842b-4e58-bf6d-83f77f19f7b9 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-changed-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-870a5631-2ce1-425e-92ca-d781f868f225 req-4ce0e604-842b-4e58-bf6d-83f77f19f7b9 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing instance network info cache due to event network-changed-8c995dcb-b43b-4bdd-b631-e3b6104d49d0. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-870a5631-2ce1-425e-92ca-d781f868f225 req-4ce0e604-842b-4e58-bf6d-83f77f19f7b9 service nova] Acquiring lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-870a5631-2ce1-425e-92ca-d781f868f225 req-4ce0e604-842b-4e58-bf6d-83f77f19f7b9 service nova] Acquired lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG nova.network.neutron [req-870a5631-2ce1-425e-92ca-d781f868f225 req-4ce0e604-842b-4e58-bf6d-83f77f19f7b9 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing network info cache for port 8c995dcb-b43b-4bdd-b631-e3b6104d49d0 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG nova.network.neutron [req-870a5631-2ce1-425e-92ca-d781f868f225 req-4ce0e604-842b-4e58-bf6d-83f77f19f7b9 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG nova.network.neutron [req-870a5631-2ce1-425e-92ca-d781f868f225 req-4ce0e604-842b-4e58-bf6d-83f77f19f7b9 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-870a5631-2ce1-425e-92ca-d781f868f225 req-4ce0e604-842b-4e58-bf6d-83f77f19f7b9 service nova] Releasing lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Successfully updated port: c9994009-b14a-4ff4-ac36-f02b0feccfb6 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:17:49 user nova-compute[71317]: INFO nova.virt.block_device [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Booting with volume 18016742-a15a-4e85-9fb9-4cbfd819316b at /dev/vdb
Apr 14 04:17:49 user nova-compute[71317]: DEBUG os_brick.utils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bf9058f4-2dbb-425b-9ef7-bc5a861a9422]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa688da0-8057-43b7-a7fa-e3e9cae07692]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[26db51c2-c601-41e2-9767-18dcc5e1c3da]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[abb5db17-e1ca-4076-ad86-219dda540839]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:17:49 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:17:49 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7267e705-268e-4cdc-9ef1-b057d0ab1887]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG os_brick.utils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] <== get_connector_properties: return (141ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:17:49 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating existing volume attachment record: e2915d95-5bab-4b56-bc25-4f938db97e78 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:17:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:50 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Successfully updated port: 11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG nova.compute.manager [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-changed-c9994009-b14a-4ff4-ac36-f02b0feccfb6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG nova.compute.manager [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing instance network info cache due to event network-changed-c9994009-b14a-4ff4-ac36-f02b0feccfb6. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] Acquiring lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] Acquired lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG nova.network.neutron [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing network info cache for port c9994009-b14a-4ff4-ac36-f02b0feccfb6 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG nova.network.neutron [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Successfully updated port: 8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG nova.network.neutron [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] Releasing lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG nova.compute.manager [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-changed-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG nova.compute.manager [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing instance network info cache due to event network-changed-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] Acquiring lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] Acquired lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG nova.network.neutron [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing network info cache for port 11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG nova.network.neutron [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG nova.network.neutron [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-fb4dd4ac-26d2-4734-b8cb-733c8cc206b2 req-9d0cdff7-8391-4dec-8f12-89a1dd7508e4 service nova] Releasing lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Successfully updated port: 07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:17:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Successfully updated port: 512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:17:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquired lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:17:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:17:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-changed-8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing instance network info cache due to event network-changed-8257c16c-6115-4b67-a739-91cb1d5e8bc6. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:17:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] Acquiring lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:17:54 user nova-compute[71317]: INFO nova.virt.block_device [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Booting with volume c8315093-aba5-4adf-8381-c59806b42282 at /dev/vdc
Apr 14 04:17:54 user nova-compute[71317]: DEBUG os_brick.utils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0c6e209-d6c6-48ad-8674-a5ee4d15bdff]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:17:54 user nova-compute[71317]: INFO nova.compute.claims [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Claim successful on node user
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[35a96fdb-92f9-4cc6-bf39-dffdf6e1ef3f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[365530d5-5638-4135-9540-b7db5012d323]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[03b5eac8-4362-402f-9bad-8ce702e4dc50]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:17:54 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:17:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4812ac23-b74a-4df9-a3d0-553dee93395e]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG os_brick.utils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] <== get_connector_properties: return (130ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:17:54 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating existing volume attachment record: a062bc6d-16a2-4670-acaf-78b62d1dfd90 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.535s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:17:55 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.policy [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '982f167333954612a51911fc8d6c1ce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a60f60717b44f4ba904a949eabfe608', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:17:55 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Creating image(s)
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "/opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "/opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "/opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "b316d97aa3eaa73df897c497fcec164014ef27e9" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "b316d97aa3eaa73df897c497fcec164014ef27e9" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating instance_info_cache with network_info: [{"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Releasing lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Instance network_info: |[{"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] Acquired lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.network.neutron [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing network info cache for port 8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9.part --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9.part --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.virt.images [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] 2921b7b4-17d1-4c56-9fbc-a30733db1472 was qcow2, converting to raw {{(pid=71317) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9.part /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9.converted {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9.part /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9.converted" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9.converted --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:55 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Successfully created port: ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9.converted --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "b316d97aa3eaa73df897c497fcec164014ef27e9" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 0.714s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updated VIF entry in instance network info cache for port 8257c16c-6115-4b67-a739-91cb1d5e8bc6. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating instance_info_cache with network_info: [{"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9 --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "b316d97aa3eaa73df897c497fcec164014ef27e9" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "b316d97aa3eaa73df897c497fcec164014ef27e9" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] Releasing lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-changed-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing instance network info cache due to event network-changed-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] Acquiring lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] Acquired lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing network info cache for port 07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9 --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9,backing_fmt=raw /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9,backing_fmt=raw /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk 1073741824" returned: 0 in 0.054s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "b316d97aa3eaa73df897c497fcec164014ef27e9" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.201s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9 --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Checking if we can resize image /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Successfully updated port: ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Cannot resize image /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.objects.instance [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'migration_context' on Instance uuid 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquired lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Ensure instance console log exists: /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-899a93fc-546c-4f6e-85ee-711fa12af6c3 req-af87fc58-dc20-40ee-b1ab-6dd9a5aae1a4 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-changed-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-899a93fc-546c-4f6e-85ee-711fa12af6c3 req-af87fc58-dc20-40ee-b1ab-6dd9a5aae1a4 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Refreshing instance network info cache due to event network-changed-ef7670a9-3421-45de-a1cd-976ddb46f2a1. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-899a93fc-546c-4f6e-85ee-711fa12af6c3 req-af87fc58-dc20-40ee-b1ab-6dd9a5aae1a4 service nova] Acquiring lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updated VIF entry in instance network info cache for port 07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating instance_info_cache with network_info: [{"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] Releasing lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-changed-512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing instance network info cache due to event network-changed-512baf47-b5bf-4cf0-8300-f0e0167fc70e. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] Acquiring lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] Acquired lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing network info cache for port 512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:17:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updating instance_info_cache with network_info: [{"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Releasing lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Instance network_info: |[{"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-899a93fc-546c-4f6e-85ee-711fa12af6c3 req-af87fc58-dc20-40ee-b1ab-6dd9a5aae1a4 service nova] Acquired lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.network.neutron [req-899a93fc-546c-4f6e-85ee-711fa12af6c3 req-af87fc58-dc20-40ee-b1ab-6dd9a5aae1a4 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Refreshing network info cache for port ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Start _get_guest_xml network_info=[{"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:17:52Z,direct_url=<?>,disk_format='qcow2',id=2921b7b4-17d1-4c56-9fbc-a30733db1472,min_disk=0,min_ram=0,name='tempest-scenario-img--1659089695',owner='0a60f60717b44f4ba904a949eabfe608',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:17:53Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '2921b7b4-17d1-4c56-9fbc-a30733db1472'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:17:57 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:17:57 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:17:52Z,direct_url=<?>,disk_format='qcow2',id=2921b7b4-17d1-4c56-9fbc-a30733db1472,min_disk=0,min_ram=0,name='tempest-scenario-img--1659089695',owner='0a60f60717b44f4ba904a949eabfe608',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:17:53Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:17:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-99049874',display_name='tempest-TestMinimumBasicScenario-server-99049874',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-99049874',id=13,image_ref='2921b7b4-17d1-4c56-9fbc-a30733db1472',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA++2BXIcEQ2v7bj136wrEst3TLhMf3Pwm5yFI2y5OTUcyHd3G9mX9nGbcm6RmV/qF9Miy1upxS4ewe90Jmk1PeD7f+WKuozsw6VgX059vi7WiBA0frzDuQmMNJN3b4XIg==',key_name='tempest-TestMinimumBasicScenario-1599913910',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0a60f60717b44f4ba904a949eabfe608',ramdisk_id='',reservation_id='r-n2gdtvof',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='2921b7b4-17d1-4c56-9fbc-a30733db1472',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1872250892',owner_user_name='tempest-TestMinimumBasicScenario-1872250892-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:55Z,user_data=None,user_id='982f167333954612a51911fc8d6c1ce9',uuid=4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converting VIF {"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.objects.instance [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'pci_devices' on Instance uuid 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:17:57 user nova-compute[71317]:   <uuid>4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e</uuid>
Apr 14 04:17:57 user nova-compute[71317]:   <name>instance-0000000d</name>
Apr 14 04:17:57 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:17:57 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:17:57 user nova-compute[71317]:   <metadata>
Apr 14 04:17:57 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:17:57 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:17:57 user nova-compute[71317]:       <nova:name>tempest-TestMinimumBasicScenario-server-99049874</nova:name>
Apr 14 04:17:57 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:17:57</nova:creationTime>
Apr 14 04:17:57 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:17:57 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:17:57 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:17:57 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:17:57 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:17:57 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:17:57 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:17:57 user nova-compute[71317]:       <nova:owner>
Apr 14 04:17:57 user nova-compute[71317]:         <nova:user uuid="982f167333954612a51911fc8d6c1ce9">tempest-TestMinimumBasicScenario-1872250892-project-member</nova:user>
Apr 14 04:17:57 user nova-compute[71317]:         <nova:project uuid="0a60f60717b44f4ba904a949eabfe608">tempest-TestMinimumBasicScenario-1872250892</nova:project>
Apr 14 04:17:57 user nova-compute[71317]:       </nova:owner>
Apr 14 04:17:57 user nova-compute[71317]:       <nova:root type="image" uuid="2921b7b4-17d1-4c56-9fbc-a30733db1472"/>
Apr 14 04:17:57 user nova-compute[71317]:       <nova:ports>
Apr 14 04:17:57 user nova-compute[71317]:         <nova:port uuid="ef7670a9-3421-45de-a1cd-976ddb46f2a1">
Apr 14 04:17:57 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Apr 14 04:17:57 user nova-compute[71317]:         </nova:port>
Apr 14 04:17:57 user nova-compute[71317]:       </nova:ports>
Apr 14 04:17:57 user nova-compute[71317]:     </nova:instance>
Apr 14 04:17:57 user nova-compute[71317]:   </metadata>
Apr 14 04:17:57 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:17:57 user nova-compute[71317]:     <system>
Apr 14 04:17:57 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:17:57 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:17:57 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:17:57 user nova-compute[71317]:       <entry name="serial">4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e</entry>
Apr 14 04:17:57 user nova-compute[71317]:       <entry name="uuid">4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e</entry>
Apr 14 04:17:57 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:17:57 user nova-compute[71317]:     </system>
Apr 14 04:17:57 user nova-compute[71317]:   </sysinfo>
Apr 14 04:17:57 user nova-compute[71317]:   <os>
Apr 14 04:17:57 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:17:57 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:17:57 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:17:57 user nova-compute[71317]:   </os>
Apr 14 04:17:57 user nova-compute[71317]:   <features>
Apr 14 04:17:57 user nova-compute[71317]:     <acpi/>
Apr 14 04:17:57 user nova-compute[71317]:     <apic/>
Apr 14 04:17:57 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:17:57 user nova-compute[71317]:   </features>
Apr 14 04:17:57 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:17:57 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:17:57 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:17:57 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:17:57 user nova-compute[71317]:   </clock>
Apr 14 04:17:57 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:17:57 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:17:57 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:17:57 user nova-compute[71317]:   </cpu>
Apr 14 04:17:57 user nova-compute[71317]:   <devices>
Apr 14 04:17:57 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:17:57 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:17:57 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk"/>
Apr 14 04:17:57 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:17:57 user nova-compute[71317]:     </disk>
Apr 14 04:17:57 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:17:57 user nova-compute[71317]:       <mac address="fa:16:3e:0f:a8:af"/>
Apr 14 04:17:57 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:17:57 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:17:57 user nova-compute[71317]:       <target dev="tapef7670a9-34"/>
Apr 14 04:17:57 user nova-compute[71317]:     </interface>
Apr 14 04:17:57 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:17:57 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/console.log" append="off"/>
Apr 14 04:17:57 user nova-compute[71317]:     </serial>
Apr 14 04:17:57 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:17:57 user nova-compute[71317]:     <video>
Apr 14 04:17:57 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:17:57 user nova-compute[71317]:     </video>
Apr 14 04:17:57 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:17:57 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:17:57 user nova-compute[71317]:     </rng>
Apr 14 04:17:57 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:17:57 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:17:57 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:17:57 user nova-compute[71317]:     </memballoon>
Apr 14 04:17:57 user nova-compute[71317]:   </devices>
Apr 14 04:17:57 user nova-compute[71317]: </domain>
Apr 14 04:17:57 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:17:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-99049874',display_name='tempest-TestMinimumBasicScenario-server-99049874',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-99049874',id=13,image_ref='2921b7b4-17d1-4c56-9fbc-a30733db1472',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA++2BXIcEQ2v7bj136wrEst3TLhMf3Pwm5yFI2y5OTUcyHd3G9mX9nGbcm6RmV/qF9Miy1upxS4ewe90Jmk1PeD7f+WKuozsw6VgX059vi7WiBA0frzDuQmMNJN3b4XIg==',key_name='tempest-TestMinimumBasicScenario-1599913910',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0a60f60717b44f4ba904a949eabfe608',ramdisk_id='',reservation_id='r-n2gdtvof',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='2921b7b4-17d1-4c56-9fbc-a30733db1472',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1872250892',owner_user_name='tempest-TestMinimumBasicScenario-1872250892-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:55Z,user_data=None,user_id='982f167333954612a51911fc8d6c1ce9',uuid=4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converting VIF {"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG os_vif [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapef7670a9-34, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapef7670a9-34, col_values=(('external_ids', {'iface-id': 'ef7670a9-3421-45de-a1cd-976ddb46f2a1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:0f:a8:af', 'vm-uuid': '4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:57 user nova-compute[71317]: INFO os_vif [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34')
Apr 14 04:17:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.network.neutron [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updated VIF entry in instance network info cache for port 512baf47-b5bf-4cf0-8300-f0e0167fc70e. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.network.neutron [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating instance_info_cache with network_info: [{"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] No VIF found with MAC fa:16:3e:0f:a8:af, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec50a640-a83c-49e7-bb88-f2c57d1a2d80 req-36587cc1-8315-4b55-9478-89e66273eb12 service nova] Releasing lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.network.neutron [req-899a93fc-546c-4f6e-85ee-711fa12af6c3 req-af87fc58-dc20-40ee-b1ab-6dd9a5aae1a4 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updated VIF entry in instance network info cache for port ef7670a9-3421-45de-a1cd-976ddb46f2a1. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG nova.network.neutron [req-899a93fc-546c-4f6e-85ee-711fa12af6c3 req-af87fc58-dc20-40ee-b1ab-6dd9a5aae1a4 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updating instance_info_cache with network_info: [{"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:17:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-899a93fc-546c-4f6e-85ee-711fa12af6c3 req-af87fc58-dc20-40ee-b1ab-6dd9a5aae1a4 service nova] Releasing lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:17:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-1024a532-9bf1-4397-b727-e186647fdbc8 req-e826de5c-3991-4e50-ab0a-49cfb36c9016 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:17:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1024a532-9bf1-4397-b727-e186647fdbc8 req-e826de5c-3991-4e50-ab0a-49cfb36c9016 service nova] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1024a532-9bf1-4397-b727-e186647fdbc8 req-e826de5c-3991-4e50-ab0a-49cfb36c9016 service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1024a532-9bf1-4397-b727-e186647fdbc8 req-e826de5c-3991-4e50-ab0a-49cfb36c9016 service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-1024a532-9bf1-4397-b727-e186647fdbc8 req-e826de5c-3991-4e50-ab0a-49cfb36c9016 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] No waiting events found dispatching network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:17:58 user nova-compute[71317]: WARNING nova.compute.manager [req-1024a532-9bf1-4397-b727-e186647fdbc8 req-e826de5c-3991-4e50-ab0a-49cfb36c9016 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received unexpected event network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 for instance with vm_state building and task_state spawning.
Apr 14 04:17:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:17:59 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Creating image(s)
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Ensure instance console log exists: /opt/stack/data/nova/instances/55ba1cae-2b2f-43de-86db-996a8791fb02/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Start _get_guest_xml network_info=[{"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk', 'boot_index': '2'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk', 'boot_index': '3'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ZFpuzxGw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '55ba1cae-2b2f-43de-86db-996a8791fb02', 'attached_at': '', 'detached_at': '', 'volume_id': '31382528-1966-4f19-8a4a-94983a6314c3', 'serial': '31382528-1966-4f19-8a4a-94983a6314c3'}, 'attachment_id': 'e342f29f-eaa0-4d55-bbe5-e2dd4fd88e97', 'volume_type': None}, {'disk_bus': 'virtio', 'boot_index': 1, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nQRoZmkl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '55ba1cae-2b2f-43de-86db-996a8791fb02', 'attached_at': '', 'detached_at': '', 'volume_id': '18016742-a15a-4e85-9fb9-4cbfd819316b', 'serial': '18016742-a15a-4e85-9fb9-4cbfd819316b'}, 'attachment_id': 'e2915d95-5bab-4b56-bc25-4f938db97e78', 'volume_type': None}, {'disk_bus': 'virtio', 'boot_index': 2, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vdc', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HjUKqOuQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '55ba1cae-2b2f-43de-86db-996a8791fb02', 'attached_at': '', 'detached_at': '', 'volume_id': 'c8315093-aba5-4adf-8381-c59806b42282', 'serial': 'c8315093-aba5-4adf-8381-c59806b42282'}, 'attachment_id': 'a062bc6d-16a2-4670-acaf-78b62d1dfd90', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:17:59 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:17:59 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "cache_volume_driver" by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "cache_volume_driver" acquired by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connector [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Factory for ISCSI on None {{(pid=71317) factory /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connector.py:281}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "cache_volume_driver" "released" by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" :: held 0.004s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ZFpuzxGw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:17:59 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:17:59 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:17:59 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260" returned: 21 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[15743c27-8e7e-45b0-ae03-aff5a9866219]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:17:59 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3858b328-05a6-4947-89f9-4320e13c9bd5]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3] added
Apr 14 04:17:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bbb7fae-f9af-4aa8-a65a-c5b74f248265]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c9cda84-93b3-4422-8c0b-a3b6727b7f47]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ZFpuzxGw {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ZFpuzxGw" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f117604c-7a59-464d-8dcd-37722e4b11b8]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'ZFpuzxGw'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8279bae1-f283-44fc-a22b-f1ef0cd21713]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a2b6e5c-750d-490b-89ae-fe536ad53190]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 14 04:17:59 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 14 04:17:59 user nova-compute[71317]:  {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:17:59 user nova-compute[71317]: WARNING os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 14 04:17:59 user nova-compute[71317]: 
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --login" returned: 0 in 0.048s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1640b2d8-f5e7-4d78-b321-ed5a80d8cc1b]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3, portal: 172.16.0.220,3260]
Apr 14 04:17:59 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3, portal: 172.16.0.220,3260] successful.
Apr 14 04:17:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3fe872ac-5927-4eb9-87bd-64df73939c93]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[458b2e0a-231b-4d49-bbfc-066a20867073]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:17:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:17:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.247s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] HCTL ('33', '-', '-', 0) found on session 2 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[db836e05-ff78-4075-9632-00d8210bb4cc]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Searching for a device in session 2 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:17:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ZFpuzxGw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445880.6940467, 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:18:00 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] VM Resumed (Lifecycle Event)
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:18:00 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Instance spawned successfully.
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:00 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445880.6957247, 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:18:00 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] VM Started (Lifecycle Event)
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:18:00 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.279s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] <== connect_volume: return (1281ms) {'type': 'block', 'scsi_wwn': '23132323161306233', 'path': '/dev/sdb'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23132323161306233', 'path': '/dev/sdb'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:18:00 user nova-compute[71317]: INFO nova.compute.manager [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Took 5.48 seconds to spawn the instance on the hypervisor.
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nQRoZmkl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:18:00 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:18:00 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:18:00 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260" returned: 21 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ead52c13-45aa-450f-bc0a-0b9857012654]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:18:00 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[85666cba-40be-4421-8775-4d6b074c471b]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b] added
Apr 14 04:18:00 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:00 user nova-compute[71317]: INFO nova.compute.manager [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Took 6.35 seconds to build instance.
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1611a0af-5fa4-4f36-9eeb-cfe97e9b2607 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.439s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[555e57d9-ff78-44ff-b235-94b2f5f732a1]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[384cd09c-a626-49d7-93e5-2ca583cdb018]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v nQRoZmkl {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v nQRoZmkl" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7f9e63a-b813-4abc-b7c3-7cc0d2d55f3d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'nQRoZmkl'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ae6a535-306c-47a9-a4ad-e23d28cdad8b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-6cb85f51-c5a3-4d60-8f81-1e74fc02b8b4 req-efde8e70-0641-4a69-929f-95dffb9c3a18 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6cb85f51-c5a3-4d60-8f81-1e74fc02b8b4 req-efde8e70-0641-4a69-929f-95dffb9c3a18 service nova] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6cb85f51-c5a3-4d60-8f81-1e74fc02b8b4 req-efde8e70-0641-4a69-929f-95dffb9c3a18 service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6cb85f51-c5a3-4d60-8f81-1e74fc02b8b4 req-efde8e70-0641-4a69-929f-95dffb9c3a18 service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-6cb85f51-c5a3-4d60-8f81-1e74fc02b8b4 req-efde8e70-0641-4a69-929f-95dffb9c3a18 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] No waiting events found dispatching network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:01 user nova-compute[71317]: WARNING nova.compute.manager [req-6cb85f51-c5a3-4d60-8f81-1e74fc02b8b4 req-efde8e70-0641-4a69-929f-95dffb9c3a18 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received unexpected event network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 for instance with vm_state active and task_state None.
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7ddd458-36c8-4190-aea4-99333b54a4bd]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --login" returned: 0 in 0.060s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea6a20e9-bf2b-4796-979a-0bda46da47b4]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b, portal: 172.16.0.220,3260]
Apr 14 04:18:01 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b, portal: 172.16.0.220,3260] successful.
Apr 14 04:18:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3684cd3-6684-4747-9695-c1cc63b527a2]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e04492b1-fb05-48a4-a14f-58536394c974]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:01 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:01 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.270s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] HCTL ('34', '-', '-', 0) found on session 3 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[94d18485-974d-4e6a-80e3-ec5d9c48f32d]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Searching for a device in session 3 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:18:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nQRoZmkl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.320s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] <== connect_volume: return (1322ms) {'type': 'block', 'scsi_wwn': '26164363435343866', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26164363435343866', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HjUKqOuQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:18:02 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:18:02 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:18:02 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[939840a4-a889-48ff-97be-c0b917b7944c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:18:02 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa816b8a-57c3-4051-a5ff-e3e6cdc05ce5]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282] added
Apr 14 04:18:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3390202-1b97-489b-8be5-62fb2f5dc407]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[59f27066-33c2-4ead-854c-4ffdf89d9854]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HjUKqOuQ {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HjUKqOuQ" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1895b54-86bd-4ec9-9c16-9b49c32d93bb]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'HjUKqOuQ'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4cb98ffe-1fb1-4bcf-b56b-d742c68cbd8c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d1ea3e7-32cf-4c3b-8c2d-ffad70f8b805]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:02 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:02 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --login" returned: 0 in 0.058s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[095e7a70-0995-4c85-81c4-dd623b000075]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282, portal: 172.16.0.220,3260]
Apr 14 04:18:02 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282, portal: 172.16.0.220,3260] successful.
Apr 14 04:18:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[62850831-ab90-44fc-8a50-937fe2e1d3d7]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[60d0feec-b5b7-494b-95df-75fc1161bdde]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:02 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:02 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:18:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:02 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:02 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:18:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.269s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] HCTL ('35', '-', '-', 0) found on session 4 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7fc23f2f-b850-4370-8853-9f2d61fd125e]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Searching for a device in session 4 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:18:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HjUKqOuQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.312s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] <== connect_volume: return (1313ms) {'type': 'block', 'scsi_wwn': '23062663337326138', 'path': '/dev/sdd'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23062663337326138', 'path': '/dev/sdd'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d1:8d:17,bridge_name='br-int',has_traffic_filtering=True,id=eea29fce-f6e3-4715-9a46-2a638462c6f8,network=Network(a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeea29fce-f6') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b3:13:af,bridge_name='br-int',has_traffic_filtering=True,id=8c995dcb-b43b-4bdd-b631-e3b6104d49d0,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8c995dcb-b4') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:86:19:cc,bridge_name='br-int',has_traffic_filtering=True,id=c9994009-b14a-4ff4-ac36-f02b0feccfb6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapc9994009-b1') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:48:3e:f5,bridge_name='br-int',has_traffic_filtering=True,id=11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap11fd6438-8a') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:95:73:7e,bridge_name='br-int',has_traffic_filtering=True,id=8257c16c-6115-4b67-a739-91cb1d5e8bc6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8257c16c-61') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9c:4f:8e,bridge_name='br-int',has_traffic_filtering=True,id=07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07a4c8ef-59') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:53:d2:74,bridge_name='br-int',has_traffic_filtering=True,id=512baf47-b5bf-4cf0-8300-f0e0167fc70e,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap512baf47-b5') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.objects.instance [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lazy-loading 'pci_devices' on Instance uuid 55ba1cae-2b2f-43de-86db-996a8791fb02 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:18:03 user nova-compute[71317]:   <uuid>55ba1cae-2b2f-43de-86db-996a8791fb02</uuid>
Apr 14 04:18:03 user nova-compute[71317]:   <name>instance-0000000c</name>
Apr 14 04:18:03 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:18:03 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:18:03 user nova-compute[71317]:   <metadata>
Apr 14 04:18:03 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:18:03 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:18:03 user nova-compute[71317]:       <nova:name>tempest-device-tagging-server-889777707</nova:name>
Apr 14 04:18:03 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:17:59</nova:creationTime>
Apr 14 04:18:03 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:18:03 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:18:03 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:18:03 user nova-compute[71317]:       <nova:owner>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:user uuid="b848680106b74275a947f316ce6311fd">tempest-TaggedBootDevicesTest_v242-581315356-project-member</nova:user>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:project uuid="225e61ca983d4f27adc61ad33c81fce4">tempest-TaggedBootDevicesTest_v242-581315356</nova:project>
Apr 14 04:18:03 user nova-compute[71317]:       </nova:owner>
Apr 14 04:18:03 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:18:03 user nova-compute[71317]:       <nova:ports>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:port uuid="eea29fce-f6e3-4715-9a46-2a638462c6f8">
Apr 14 04:18:03 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 04:18:03 user nova-compute[71317]:         </nova:port>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:port uuid="8c995dcb-b43b-4bdd-b631-e3b6104d49d0">
Apr 14 04:18:03 user nova-compute[71317]:           <nova:ip type="fixed" address="10.1.1.66" ipVersion="4"/>
Apr 14 04:18:03 user nova-compute[71317]:         </nova:port>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:port uuid="c9994009-b14a-4ff4-ac36-f02b0feccfb6">
Apr 14 04:18:03 user nova-compute[71317]:           <nova:ip type="fixed" address="10.1.1.209" ipVersion="4"/>
Apr 14 04:18:03 user nova-compute[71317]:         </nova:port>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:port uuid="11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe">
Apr 14 04:18:03 user nova-compute[71317]:           <nova:ip type="fixed" address="10.1.1.147" ipVersion="4"/>
Apr 14 04:18:03 user nova-compute[71317]:         </nova:port>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:port uuid="8257c16c-6115-4b67-a739-91cb1d5e8bc6">
Apr 14 04:18:03 user nova-compute[71317]:           <nova:ip type="fixed" address="10.1.1.153" ipVersion="4"/>
Apr 14 04:18:03 user nova-compute[71317]:         </nova:port>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:port uuid="07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5">
Apr 14 04:18:03 user nova-compute[71317]:           <nova:ip type="fixed" address="10.2.2.100" ipVersion="4"/>
Apr 14 04:18:03 user nova-compute[71317]:         </nova:port>
Apr 14 04:18:03 user nova-compute[71317]:         <nova:port uuid="512baf47-b5bf-4cf0-8300-f0e0167fc70e">
Apr 14 04:18:03 user nova-compute[71317]:           <nova:ip type="fixed" address="10.2.2.200" ipVersion="4"/>
Apr 14 04:18:03 user nova-compute[71317]:         </nova:port>
Apr 14 04:18:03 user nova-compute[71317]:       </nova:ports>
Apr 14 04:18:03 user nova-compute[71317]:     </nova:instance>
Apr 14 04:18:03 user nova-compute[71317]:   </metadata>
Apr 14 04:18:03 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:18:03 user nova-compute[71317]:     <system>
Apr 14 04:18:03 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:18:03 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:18:03 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:18:03 user nova-compute[71317]:       <entry name="serial">55ba1cae-2b2f-43de-86db-996a8791fb02</entry>
Apr 14 04:18:03 user nova-compute[71317]:       <entry name="uuid">55ba1cae-2b2f-43de-86db-996a8791fb02</entry>
Apr 14 04:18:03 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:18:03 user nova-compute[71317]:     </system>
Apr 14 04:18:03 user nova-compute[71317]:   </sysinfo>
Apr 14 04:18:03 user nova-compute[71317]:   <os>
Apr 14 04:18:03 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:18:03 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:18:03 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:18:03 user nova-compute[71317]:   </os>
Apr 14 04:18:03 user nova-compute[71317]:   <features>
Apr 14 04:18:03 user nova-compute[71317]:     <acpi/>
Apr 14 04:18:03 user nova-compute[71317]:     <apic/>
Apr 14 04:18:03 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:18:03 user nova-compute[71317]:   </features>
Apr 14 04:18:03 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:18:03 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:18:03 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:18:03 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:18:03 user nova-compute[71317]:   </clock>
Apr 14 04:18:03 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:18:03 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:18:03 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:18:03 user nova-compute[71317]:   </cpu>
Apr 14 04:18:03 user nova-compute[71317]:   <devices>
Apr 14 04:18:03 user nova-compute[71317]:     <disk type="file" device="cdrom">
Apr 14 04:18:03 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none"/>
Apr 14 04:18:03 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/55ba1cae-2b2f-43de-86db-996a8791fb02/disk.config"/>
Apr 14 04:18:03 user nova-compute[71317]:       <target dev="hda" bus="ide"/>
Apr 14 04:18:03 user nova-compute[71317]:     </disk>
Apr 14 04:18:03 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:18:03 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:18:03 user nova-compute[71317]:       <source dev="/dev/sdb"/>
Apr 14 04:18:03 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:18:03 user nova-compute[71317]:       <serial>31382528-1966-4f19-8a4a-94983a6314c3</serial>
Apr 14 04:18:03 user nova-compute[71317]:     </disk>
Apr 14 04:18:03 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:18:03 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:18:03 user nova-compute[71317]:       <source dev="/dev/sdc"/>
Apr 14 04:18:03 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:18:03 user nova-compute[71317]:       <serial>18016742-a15a-4e85-9fb9-4cbfd819316b</serial>
Apr 14 04:18:03 user nova-compute[71317]:     </disk>
Apr 14 04:18:03 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:18:03 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:18:03 user nova-compute[71317]:       <source dev="/dev/sdd"/>
Apr 14 04:18:03 user nova-compute[71317]:       <target dev="vdc" bus="virtio"/>
Apr 14 04:18:03 user nova-compute[71317]:       <serial>c8315093-aba5-4adf-8381-c59806b42282</serial>
Apr 14 04:18:03 user nova-compute[71317]:     </disk>
Apr 14 04:18:03 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:18:03 user nova-compute[71317]:       <mac address="fa:16:3e:d1:8d:17"/>
Apr 14 04:18:03 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:18:03 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:18:03 user nova-compute[71317]:       <target dev="tapeea29fce-f6"/>
Apr 14 04:18:03 user nova-compute[71317]:     </interface>
Apr 14 04:18:03 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:18:03 user nova-compute[71317]:       <mac address="fa:16:3e:b3:13:af"/>
Apr 14 04:18:03 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:18:03 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:18:03 user nova-compute[71317]:       <target dev="tap8c995dcb-b4"/>
Apr 14 04:18:03 user nova-compute[71317]:     </interface>
Apr 14 04:18:03 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:18:03 user nova-compute[71317]:       <mac address="fa:16:3e:86:19:cc"/>
Apr 14 04:18:03 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:18:03 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:18:03 user nova-compute[71317]:       <target dev="tapc9994009-b1"/>
Apr 14 04:18:03 user nova-compute[71317]:     </interface>
Apr 14 04:18:03 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:18:03 user nova-compute[71317]:       <mac address="fa:16:3e:48:3e:f5"/>
Apr 14 04:18:03 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:18:03 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:18:03 user nova-compute[71317]:       <target dev="tap11fd6438-8a"/>
Apr 14 04:18:03 user nova-compute[71317]:     </interface>
Apr 14 04:18:03 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:18:03 user nova-compute[71317]:       <mac address="fa:16:3e:95:73:7e"/>
Apr 14 04:18:03 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:18:03 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:18:03 user nova-compute[71317]:       <target dev="tap8257c16c-61"/>
Apr 14 04:18:03 user nova-compute[71317]:     </interface>
Apr 14 04:18:03 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:18:03 user nova-compute[71317]:       <mac address="fa:16:3e:9c:4f:8e"/>
Apr 14 04:18:03 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:18:03 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:18:03 user nova-compute[71317]:       <target dev="tap07a4c8ef-59"/>
Apr 14 04:18:03 user nova-compute[71317]:     </interface>
Apr 14 04:18:03 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:18:03 user nova-compute[71317]:       <mac address="fa:16:3e:53:d2:74"/>
Apr 14 04:18:03 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:18:03 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:18:03 user nova-compute[71317]:       <target dev="tap512baf47-b5"/>
Apr 14 04:18:03 user nova-compute[71317]:     </interface>
Apr 14 04:18:03 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:18:03 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/55ba1cae-2b2f-43de-86db-996a8791fb02/console.log" append="off"/>
Apr 14 04:18:03 user nova-compute[71317]:     </serial>
Apr 14 04:18:03 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:18:03 user nova-compute[71317]:     <video>
Apr 14 04:18:03 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:18:03 user nova-compute[71317]:     </video>
Apr 14 04:18:03 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:18:03 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:18:03 user nova-compute[71317]:     </rng>
Apr 14 04:18:03 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:18:03 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:18:03 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:18:03 user nova-compute[71317]:     </memballoon>
Apr 14 04:18:03 user nova-compute[71317]:   </devices>
Apr 14 04:18:03 user nova-compute[71317]: </domain>
Apr 14 04:18:03 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d1:8d:17,bridge_name='br-int',has_traffic_filtering=True,id=eea29fce-f6e3-4715-9a46-2a638462c6f8,network=Network(a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeea29fce-f6') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d1:8d:17,bridge_name='br-int',has_traffic_filtering=True,id=eea29fce-f6e3-4715-9a46-2a638462c6f8,network=Network(a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeea29fce-f6') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapeea29fce-f6, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapeea29fce-f6, col_values=(('external_ids', {'iface-id': 'eea29fce-f6e3-4715-9a46-2a638462c6f8', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d1:8d:17', 'vm-uuid': '55ba1cae-2b2f-43de-86db-996a8791fb02'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: INFO os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d1:8d:17,bridge_name='br-int',has_traffic_filtering=True,id=eea29fce-f6e3-4715-9a46-2a638462c6f8,network=Network(a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeea29fce-f6')
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b3:13:af,bridge_name='br-int',has_traffic_filtering=True,id=8c995dcb-b43b-4bdd-b631-e3b6104d49d0,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8c995dcb-b4') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b3:13:af,bridge_name='br-int',has_traffic_filtering=True,id=8c995dcb-b43b-4bdd-b631-e3b6104d49d0,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8c995dcb-b4') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8c995dcb-b4, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap8c995dcb-b4, col_values=(('external_ids', {'iface-id': '8c995dcb-b43b-4bdd-b631-e3b6104d49d0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b3:13:af', 'vm-uuid': '55ba1cae-2b2f-43de-86db-996a8791fb02'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: INFO os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b3:13:af,bridge_name='br-int',has_traffic_filtering=True,id=8c995dcb-b43b-4bdd-b631-e3b6104d49d0,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8c995dcb-b4')
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:86:19:cc,bridge_name='br-int',has_traffic_filtering=True,id=c9994009-b14a-4ff4-ac36-f02b0feccfb6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapc9994009-b1') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:86:19:cc,bridge_name='br-int',has_traffic_filtering=True,id=c9994009-b14a-4ff4-ac36-f02b0feccfb6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapc9994009-b1') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc9994009-b1, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc9994009-b1, col_values=(('external_ids', {'iface-id': 'c9994009-b14a-4ff4-ac36-f02b0feccfb6', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:86:19:cc', 'vm-uuid': '55ba1cae-2b2f-43de-86db-996a8791fb02'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: INFO os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:86:19:cc,bridge_name='br-int',has_traffic_filtering=True,id=c9994009-b14a-4ff4-ac36-f02b0feccfb6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapc9994009-b1')
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:48:3e:f5,bridge_name='br-int',has_traffic_filtering=True,id=11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap11fd6438-8a') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:48:3e:f5,bridge_name='br-int',has_traffic_filtering=True,id=11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap11fd6438-8a') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap11fd6438-8a, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap11fd6438-8a, col_values=(('external_ids', {'iface-id': '11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:48:3e:f5', 'vm-uuid': '55ba1cae-2b2f-43de-86db-996a8791fb02'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: INFO os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:48:3e:f5,bridge_name='br-int',has_traffic_filtering=True,id=11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap11fd6438-8a')
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:95:73:7e,bridge_name='br-int',has_traffic_filtering=True,id=8257c16c-6115-4b67-a739-91cb1d5e8bc6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8257c16c-61') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:95:73:7e,bridge_name='br-int',has_traffic_filtering=True,id=8257c16c-6115-4b67-a739-91cb1d5e8bc6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8257c16c-61') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8257c16c-61, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap8257c16c-61, col_values=(('external_ids', {'iface-id': '8257c16c-6115-4b67-a739-91cb1d5e8bc6', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:95:73:7e', 'vm-uuid': '55ba1cae-2b2f-43de-86db-996a8791fb02'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: INFO os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:95:73:7e,bridge_name='br-int',has_traffic_filtering=True,id=8257c16c-6115-4b67-a739-91cb1d5e8bc6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8257c16c-61')
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9c:4f:8e,bridge_name='br-int',has_traffic_filtering=True,id=07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07a4c8ef-59') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9c:4f:8e,bridge_name='br-int',has_traffic_filtering=True,id=07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07a4c8ef-59') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap07a4c8ef-59, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap07a4c8ef-59, col_values=(('external_ids', {'iface-id': '07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:9c:4f:8e', 'vm-uuid': '55ba1cae-2b2f-43de-86db-996a8791fb02'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: INFO os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9c:4f:8e,bridge_name='br-int',has_traffic_filtering=True,id=07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07a4c8ef-59')
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:53:d2:74,bridge_name='br-int',has_traffic_filtering=True,id=512baf47-b5bf-4cf0-8300-f0e0167fc70e,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap512baf47-b5') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:53:d2:74,bridge_name='br-int',has_traffic_filtering=True,id=512baf47-b5bf-4cf0-8300-f0e0167fc70e,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap512baf47-b5') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap512baf47-b5, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap512baf47-b5, col_values=(('external_ids', {'iface-id': '512baf47-b5bf-4cf0-8300-f0e0167fc70e', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:53:d2:74', 'vm-uuid': '55ba1cae-2b2f-43de-86db-996a8791fb02'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:03 user nova-compute[71317]: INFO os_vif [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:53:d2:74,bridge_name='br-int',has_traffic_filtering=True,id=512baf47-b5bf-4cf0-8300-f0e0167fc70e,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap512baf47-b5')
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] No BDM found with device name hda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] No BDM found with device name vdc, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] No VIF found with MAC fa:16:3e:d1:8d:17, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:18:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] No VIF found with MAC fa:16:3e:95:73:7e, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:18:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Using config drive
Apr 14 04:18:04 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Creating config drive at /opt/stack/data/nova/instances/55ba1cae-2b2f-43de-86db-996a8791fb02/disk.config
Apr 14 04:18:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/55ba1cae-2b2f-43de-86db-996a8791fb02/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmp_35vlnos {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/55ba1cae-2b2f-43de-86db-996a8791fb02/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmp_35vlnos" returned: 0 in 0.064s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:05 user nova-compute[71317]: DEBUG nova.compute.manager [req-c53af1c0-d59b-431e-a4b6-acd25d1f5812 req-38090454-1880-488a-be9c-1e47f41186d1 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c53af1c0-d59b-431e-a4b6-acd25d1f5812 req-38090454-1880-488a-be9c-1e47f41186d1 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c53af1c0-d59b-431e-a4b6-acd25d1f5812 req-38090454-1880-488a-be9c-1e47f41186d1 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c53af1c0-d59b-431e-a4b6-acd25d1f5812 req-38090454-1880-488a-be9c-1e47f41186d1 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:05 user nova-compute[71317]: DEBUG nova.compute.manager [req-c53af1c0-d59b-431e-a4b6-acd25d1f5812 req-38090454-1880-488a-be9c-1e47f41186d1 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:05 user nova-compute[71317]: WARNING nova.compute.manager [req-c53af1c0-d59b-431e-a4b6-acd25d1f5812 req-38090454-1880-488a-be9c-1e47f41186d1 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-eea29fce-f6e3-4715-9a46-2a638462c6f8 for instance with vm_state building and task_state spawning.
Apr 14 04:18:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-710ddf74-9924-4770-842d-6f99133c1ded req-a81f3127-a08d-4ec7-9a54-64dadbed7fb1 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-710ddf74-9924-4770-842d-6f99133c1ded req-a81f3127-a08d-4ec7-9a54-64dadbed7fb1 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-710ddf74-9924-4770-842d-6f99133c1ded req-a81f3127-a08d-4ec7-9a54-64dadbed7fb1 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-710ddf74-9924-4770-842d-6f99133c1ded req-a81f3127-a08d-4ec7-9a54-64dadbed7fb1 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-710ddf74-9924-4770-842d-6f99133c1ded req-a81f3127-a08d-4ec7-9a54-64dadbed7fb1 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:06 user nova-compute[71317]: WARNING nova.compute.manager [req-710ddf74-9924-4770-842d-6f99133c1ded req-a81f3127-a08d-4ec7-9a54-64dadbed7fb1 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 for instance with vm_state building and task_state spawning.
Apr 14 04:18:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:08 user nova-compute[71317]: WARNING nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-eea29fce-f6e3-4715-9a46-2a638462c6f8 for instance with vm_state building and task_state spawning.
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:08 user nova-compute[71317]: WARNING nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe for instance with vm_state building and task_state spawning.
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:08 user nova-compute[71317]: WARNING nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe for instance with vm_state building and task_state spawning.
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:08 user nova-compute[71317]: WARNING nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-8257c16c-6115-4b67-a739-91cb1d5e8bc6 for instance with vm_state building and task_state spawning.
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:08 user nova-compute[71317]: WARNING nova.compute.manager [req-5214217c-f508-4f94-9d3e-f01b3adcac45 req-4f0bfc3a-9383-45be-a57f-179507cc1b31 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-8257c16c-6115-4b67-a739-91cb1d5e8bc6 for instance with vm_state building and task_state spawning.
Apr 14 04:18:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:08 user nova-compute[71317]: WARNING nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 for instance with vm_state building and task_state spawning.
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-c9994009-b14a-4ff4-ac36-f02b0feccfb6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-c9994009-b14a-4ff4-ac36-f02b0feccfb6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:08 user nova-compute[71317]: WARNING nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-c9994009-b14a-4ff4-ac36-f02b0feccfb6 for instance with vm_state building and task_state spawning.
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-c9994009-b14a-4ff4-ac36-f02b0feccfb6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-c9994009-b14a-4ff4-ac36-f02b0feccfb6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:08 user nova-compute[71317]: WARNING nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-c9994009-b14a-4ff4-ac36-f02b0feccfb6 for instance with vm_state building and task_state spawning.
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:08 user nova-compute[71317]: WARNING nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 for instance with vm_state building and task_state spawning.
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:08 user nova-compute[71317]: WARNING nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 for instance with vm_state building and task_state spawning.
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:08 user nova-compute[71317]: WARNING nova.compute.manager [req-5e1774a6-6ed7-4e79-b5f7-a3cd32ba837a req-4f433a41-25e1-4024-b82f-b1b45f0538e6 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-512baf47-b5bf-4cf0-8300-f0e0167fc70e for instance with vm_state building and task_state spawning.
Apr 14 04:18:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445889.8006275, 55ba1cae-2b2f-43de-86db-996a8791fb02 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:18:09 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] VM Resumed (Lifecycle Event)
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:18:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Instance spawned successfully.
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:09 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445889.8058598, 55ba1cae-2b2f-43de-86db-996a8791fb02 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:18:09 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] VM Started (Lifecycle Event)
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:18:09 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:18:09 user nova-compute[71317]: INFO nova.compute.manager [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Took 10.45 seconds to spawn the instance on the hypervisor.
Apr 14 04:18:09 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:10 user nova-compute[71317]: INFO nova.compute.manager [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Took 27.62 seconds to build instance.
Apr 14 04:18:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bdd69aae-d460-4b88-b94d-0194c85d9ca9 tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 27.723s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-c67771ea-f4c9-46f7-a711-3a3f862d429e req-04885799-c5ef-4646-83df-bfd29c076639 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c67771ea-f4c9-46f7-a711-3a3f862d429e req-04885799-c5ef-4646-83df-bfd29c076639 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c67771ea-f4c9-46f7-a711-3a3f862d429e req-04885799-c5ef-4646-83df-bfd29c076639 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c67771ea-f4c9-46f7-a711-3a3f862d429e req-04885799-c5ef-4646-83df-bfd29c076639 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-c67771ea-f4c9-46f7-a711-3a3f862d429e req-04885799-c5ef-4646-83df-bfd29c076639 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:10 user nova-compute[71317]: WARNING nova.compute.manager [req-c67771ea-f4c9-46f7-a711-3a3f862d429e req-04885799-c5ef-4646-83df-bfd29c076639 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-512baf47-b5bf-4cf0-8300-f0e0167fc70e for instance with vm_state active and task_state None.
Apr 14 04:18:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "0ff97860-2441-437c-b118-6089c3cc4160" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "0ff97860-2441-437c-b118-6089c3cc4160" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "0ff97860-2441-437c-b118-6089c3cc4160-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "0ff97860-2441-437c-b118-6089c3cc4160-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "0ff97860-2441-437c-b118-6089c3cc4160-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:12 user nova-compute[71317]: INFO nova.compute.manager [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Terminating instance
Apr 14 04:18:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:18:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-573647d7-3dd6-431d-9a28-3bdfb4aff20c req-8765b5cd-9d2b-407f-a254-e9c88fa4513f service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Received event network-vif-unplugged-22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-573647d7-3dd6-431d-9a28-3bdfb4aff20c req-8765b5cd-9d2b-407f-a254-e9c88fa4513f service nova] Acquiring lock "0ff97860-2441-437c-b118-6089c3cc4160-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-573647d7-3dd6-431d-9a28-3bdfb4aff20c req-8765b5cd-9d2b-407f-a254-e9c88fa4513f service nova] Lock "0ff97860-2441-437c-b118-6089c3cc4160-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-573647d7-3dd6-431d-9a28-3bdfb4aff20c req-8765b5cd-9d2b-407f-a254-e9c88fa4513f service nova] Lock "0ff97860-2441-437c-b118-6089c3cc4160-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-573647d7-3dd6-431d-9a28-3bdfb4aff20c req-8765b5cd-9d2b-407f-a254-e9c88fa4513f service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] No waiting events found dispatching network-vif-unplugged-22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-573647d7-3dd6-431d-9a28-3bdfb4aff20c req-8765b5cd-9d2b-407f-a254-e9c88fa4513f service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Received event network-vif-unplugged-22212b8b-6656-4d88-ad5c-e0d8f657062f for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:18:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Instance destroyed successfully.
Apr 14 04:18:13 user nova-compute[71317]: DEBUG nova.objects.instance [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lazy-loading 'resources' on Instance uuid 0ff97860-2441-437c-b118-6089c3cc4160 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-844517641',display_name='tempest-VolumesActionsTest-instance-844517641',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-844517641',id=1,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:16:39Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='280aa55745a749b7baa134334d25c9c3',ramdisk_id='',reservation_id='r-bl93w3ra',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-447148503',owner_user_name='tempest-VolumesActionsTest-447148503-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:16:40Z,user_data=None,user_id='6e422970cd8340bf996a135088094f51',uuid=0ff97860-2441-437c-b118-6089c3cc4160,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "address": "fa:16:3e:41:ed:a5", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22212b8b-66", "ovs_interfaceid": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Converting VIF {"id": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "address": "fa:16:3e:41:ed:a5", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22212b8b-66", "ovs_interfaceid": "22212b8b-6656-4d88-ad5c-e0d8f657062f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:41:ed:a5,bridge_name='br-int',has_traffic_filtering=True,id=22212b8b-6656-4d88-ad5c-e0d8f657062f,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22212b8b-66') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG os_vif [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:41:ed:a5,bridge_name='br-int',has_traffic_filtering=True,id=22212b8b-6656-4d88-ad5c-e0d8f657062f,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22212b8b-66') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap22212b8b-66, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:13 user nova-compute[71317]: INFO os_vif [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:41:ed:a5,bridge_name='br-int',has_traffic_filtering=True,id=22212b8b-6656-4d88-ad5c-e0d8f657062f,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22212b8b-66')
Apr 14 04:18:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Deleting instance files /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160_del
Apr 14 04:18:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Deletion of /opt/stack/data/nova/instances/0ff97860-2441-437c-b118-6089c3cc4160_del complete
Apr 14 04:18:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Checking UEFI support for host arch (x86_64) {{(pid=71317) supports_uefi /opt/stack/nova/nova/virt/libvirt/host.py:1722}}
Apr 14 04:18:13 user nova-compute[71317]: INFO nova.virt.libvirt.host [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] UEFI support detected
Apr 14 04:18:13 user nova-compute[71317]: INFO nova.compute.manager [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Took 1.03 seconds to destroy the instance on the hypervisor.
Apr 14 04:18:13 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:18:13 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:18:14 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:14 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Took 0.99 seconds to deallocate network for instance.
Apr 14 04:18:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-f4e59366-b2f6-457b-a7fe-308e961e7fc0 req-c1b5bfc5-5c26-44aa-988e-430f9808fe3b service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Received event network-vif-deleted-22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:14 user nova-compute[71317]: INFO nova.compute.manager [req-f4e59366-b2f6-457b-a7fe-308e961e7fc0 req-c1b5bfc5-5c26-44aa-988e-430f9808fe3b service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Neutron deleted interface 22212b8b-6656-4d88-ad5c-e0d8f657062f; detaching it from the instance and deleting it from the info cache
Apr 14 04:18:14 user nova-compute[71317]: DEBUG nova.network.neutron [req-f4e59366-b2f6-457b-a7fe-308e961e7fc0 req-c1b5bfc5-5c26-44aa-988e-430f9808fe3b service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-f4e59366-b2f6-457b-a7fe-308e961e7fc0 req-c1b5bfc5-5c26-44aa-988e-430f9808fe3b service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Detach interface failed, port_id=22212b8b-6656-4d88-ad5c-e0d8f657062f, reason: Instance 0ff97860-2441-437c-b118-6089c3cc4160 could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:18:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-ac76bf17-7864-45d2-ad83-545b40554be6 req-95bb3bb9-8c76-4d22-a3b9-b99922448715 service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Received event network-vif-plugged-22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ac76bf17-7864-45d2-ad83-545b40554be6 req-95bb3bb9-8c76-4d22-a3b9-b99922448715 service nova] Acquiring lock "0ff97860-2441-437c-b118-6089c3cc4160-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ac76bf17-7864-45d2-ad83-545b40554be6 req-95bb3bb9-8c76-4d22-a3b9-b99922448715 service nova] Lock "0ff97860-2441-437c-b118-6089c3cc4160-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ac76bf17-7864-45d2-ad83-545b40554be6 req-95bb3bb9-8c76-4d22-a3b9-b99922448715 service nova] Lock "0ff97860-2441-437c-b118-6089c3cc4160-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-ac76bf17-7864-45d2-ad83-545b40554be6 req-95bb3bb9-8c76-4d22-a3b9-b99922448715 service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] No waiting events found dispatching network-vif-plugged-22212b8b-6656-4d88-ad5c-e0d8f657062f {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:15 user nova-compute[71317]: WARNING nova.compute.manager [req-ac76bf17-7864-45d2-ad83-545b40554be6 req-95bb3bb9-8c76-4d22-a3b9-b99922448715 service nova] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Received unexpected event network-vif-plugged-22212b8b-6656-4d88-ad5c-e0d8f657062f for instance with vm_state deleted and task_state None.
Apr 14 04:18:15 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:18:15 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:18:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.428s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:15 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Deleted allocations for instance 0ff97860-2441-437c-b118-6089c3cc4160
Apr 14 04:18:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-25042ad5-ce9e-4629-9bf2-ec804a898542 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "0ff97860-2441-437c-b118-6089c3cc4160" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.728s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "59e547de-a74b-4ccd-90de-e7b61cba6964" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:18:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:18:16 user nova-compute[71317]: INFO nova.compute.claims [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Claim successful on node user
Apr 14 04:18:16 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:18:16 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:18:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.481s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG nova.network.neutron [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:18:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:18:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG nova.policy [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ea68fb139b004f748cf1bcfb8d675a2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bae0fb4af555422a847620627dffeae8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:18:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Creating image(s)
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "/opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "/opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "/opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.145s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk 1073741824" returned: 0 in 0.050s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.202s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Checking if we can resize image /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Cannot resize image /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lazy-loading 'migration_context' on Instance uuid 59e547de-a74b-4ccd-90de-e7b61cba6964 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Ensure instance console log exists: /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:17 user nova-compute[71317]: DEBUG nova.network.neutron [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Successfully created port: de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Successfully updated port: de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "refresh_cache-59e547de-a74b-4ccd-90de-e7b61cba6964" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquired lock "refresh_cache-59e547de-a74b-4ccd-90de-e7b61cba6964" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-0a7f9130-472a-4acd-9c2a-35e8b0679851 req-5f22ed02-b7a8-4f5c-86e2-1bcee8552b38 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Received event network-changed-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-0a7f9130-472a-4acd-9c2a-35e8b0679851 req-5f22ed02-b7a8-4f5c-86e2-1bcee8552b38 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Refreshing instance network info cache due to event network-changed-de9378d0-c63f-48c3-9a25-e7d82b8c0f99. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0a7f9130-472a-4acd-9c2a-35e8b0679851 req-5f22ed02-b7a8-4f5c-86e2-1bcee8552b38 service nova] Acquiring lock "refresh_cache-59e547de-a74b-4ccd-90de-e7b61cba6964" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Updating instance_info_cache with network_info: [{"id": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "address": "fa:16:3e:95:29:3f", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde9378d0-c6", "ovs_interfaceid": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Releasing lock "refresh_cache-59e547de-a74b-4ccd-90de-e7b61cba6964" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Instance network_info: |[{"id": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "address": "fa:16:3e:95:29:3f", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde9378d0-c6", "ovs_interfaceid": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:18:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0a7f9130-472a-4acd-9c2a-35e8b0679851 req-5f22ed02-b7a8-4f5c-86e2-1bcee8552b38 service nova] Acquired lock "refresh_cache-59e547de-a74b-4ccd-90de-e7b61cba6964" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.network.neutron [req-0a7f9130-472a-4acd-9c2a-35e8b0679851 req-5f22ed02-b7a8-4f5c-86e2-1bcee8552b38 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Refreshing network info cache for port de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Start _get_guest_xml network_info=[{"id": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "address": "fa:16:3e:95:29:3f", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde9378d0-c6", "ovs_interfaceid": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:18:19 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:18:19 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:18:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1774642635',display_name='tempest-ServersNegativeTestJSON-server-1774642635',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1774642635',id=14,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bae0fb4af555422a847620627dffeae8',ramdisk_id='',reservation_id='r-46d3609f',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-406789078',owner_user_name='tempest-ServersNegativeTestJSON-406789078-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:18:17Z,user_data=None,user_id='ea68fb139b004f748cf1bcfb8d675a2e',uuid=59e547de-a74b-4ccd-90de-e7b61cba6964,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "address": "fa:16:3e:95:29:3f", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde9378d0-c6", "ovs_interfaceid": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Converting VIF {"id": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "address": "fa:16:3e:95:29:3f", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde9378d0-c6", "ovs_interfaceid": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:95:29:3f,bridge_name='br-int',has_traffic_filtering=True,id=de9378d0-c63f-48c3-9a25-e7d82b8c0f99,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapde9378d0-c6') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.objects.instance [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lazy-loading 'pci_devices' on Instance uuid 59e547de-a74b-4ccd-90de-e7b61cba6964 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:18:19 user nova-compute[71317]:   <uuid>59e547de-a74b-4ccd-90de-e7b61cba6964</uuid>
Apr 14 04:18:19 user nova-compute[71317]:   <name>instance-0000000e</name>
Apr 14 04:18:19 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:18:19 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:18:19 user nova-compute[71317]:   <metadata>
Apr 14 04:18:19 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:18:19 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:18:19 user nova-compute[71317]:       <nova:name>tempest-ServersNegativeTestJSON-server-1774642635</nova:name>
Apr 14 04:18:19 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:18:19</nova:creationTime>
Apr 14 04:18:19 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:18:19 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:18:19 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:18:19 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:18:19 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:18:19 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:18:19 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:18:19 user nova-compute[71317]:       <nova:owner>
Apr 14 04:18:19 user nova-compute[71317]:         <nova:user uuid="ea68fb139b004f748cf1bcfb8d675a2e">tempest-ServersNegativeTestJSON-406789078-project-member</nova:user>
Apr 14 04:18:19 user nova-compute[71317]:         <nova:project uuid="bae0fb4af555422a847620627dffeae8">tempest-ServersNegativeTestJSON-406789078</nova:project>
Apr 14 04:18:19 user nova-compute[71317]:       </nova:owner>
Apr 14 04:18:19 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:18:19 user nova-compute[71317]:       <nova:ports>
Apr 14 04:18:19 user nova-compute[71317]:         <nova:port uuid="de9378d0-c63f-48c3-9a25-e7d82b8c0f99">
Apr 14 04:18:19 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Apr 14 04:18:19 user nova-compute[71317]:         </nova:port>
Apr 14 04:18:19 user nova-compute[71317]:       </nova:ports>
Apr 14 04:18:19 user nova-compute[71317]:     </nova:instance>
Apr 14 04:18:19 user nova-compute[71317]:   </metadata>
Apr 14 04:18:19 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:18:19 user nova-compute[71317]:     <system>
Apr 14 04:18:19 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:18:19 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:18:19 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:18:19 user nova-compute[71317]:       <entry name="serial">59e547de-a74b-4ccd-90de-e7b61cba6964</entry>
Apr 14 04:18:19 user nova-compute[71317]:       <entry name="uuid">59e547de-a74b-4ccd-90de-e7b61cba6964</entry>
Apr 14 04:18:19 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:18:19 user nova-compute[71317]:     </system>
Apr 14 04:18:19 user nova-compute[71317]:   </sysinfo>
Apr 14 04:18:19 user nova-compute[71317]:   <os>
Apr 14 04:18:19 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:18:19 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:18:19 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:18:19 user nova-compute[71317]:   </os>
Apr 14 04:18:19 user nova-compute[71317]:   <features>
Apr 14 04:18:19 user nova-compute[71317]:     <acpi/>
Apr 14 04:18:19 user nova-compute[71317]:     <apic/>
Apr 14 04:18:19 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:18:19 user nova-compute[71317]:   </features>
Apr 14 04:18:19 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:18:19 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:18:19 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:18:19 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:18:19 user nova-compute[71317]:   </clock>
Apr 14 04:18:19 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:18:19 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:18:19 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:18:19 user nova-compute[71317]:   </cpu>
Apr 14 04:18:19 user nova-compute[71317]:   <devices>
Apr 14 04:18:19 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:18:19 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:18:19 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk"/>
Apr 14 04:18:19 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:18:19 user nova-compute[71317]:     </disk>
Apr 14 04:18:19 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:18:19 user nova-compute[71317]:       <mac address="fa:16:3e:95:29:3f"/>
Apr 14 04:18:19 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:18:19 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:18:19 user nova-compute[71317]:       <target dev="tapde9378d0-c6"/>
Apr 14 04:18:19 user nova-compute[71317]:     </interface>
Apr 14 04:18:19 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:18:19 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/console.log" append="off"/>
Apr 14 04:18:19 user nova-compute[71317]:     </serial>
Apr 14 04:18:19 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:18:19 user nova-compute[71317]:     <video>
Apr 14 04:18:19 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:18:19 user nova-compute[71317]:     </video>
Apr 14 04:18:19 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:18:19 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:18:19 user nova-compute[71317]:     </rng>
Apr 14 04:18:19 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:18:19 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:18:19 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:18:19 user nova-compute[71317]:     </memballoon>
Apr 14 04:18:19 user nova-compute[71317]:   </devices>
Apr 14 04:18:19 user nova-compute[71317]: </domain>
Apr 14 04:18:19 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:18:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1774642635',display_name='tempest-ServersNegativeTestJSON-server-1774642635',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1774642635',id=14,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bae0fb4af555422a847620627dffeae8',ramdisk_id='',reservation_id='r-46d3609f',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-406789078',owner_user_name='tempest-ServersNegativeTestJSON-406789078-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:18:17Z,user_data=None,user_id='ea68fb139b004f748cf1bcfb8d675a2e',uuid=59e547de-a74b-4ccd-90de-e7b61cba6964,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "address": "fa:16:3e:95:29:3f", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde9378d0-c6", "ovs_interfaceid": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Converting VIF {"id": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "address": "fa:16:3e:95:29:3f", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde9378d0-c6", "ovs_interfaceid": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:95:29:3f,bridge_name='br-int',has_traffic_filtering=True,id=de9378d0-c63f-48c3-9a25-e7d82b8c0f99,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapde9378d0-c6') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG os_vif [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:95:29:3f,bridge_name='br-int',has_traffic_filtering=True,id=de9378d0-c63f-48c3-9a25-e7d82b8c0f99,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapde9378d0-c6') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapde9378d0-c6, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapde9378d0-c6, col_values=(('external_ids', {'iface-id': 'de9378d0-c63f-48c3-9a25-e7d82b8c0f99', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:95:29:3f', 'vm-uuid': '59e547de-a74b-4ccd-90de-e7b61cba6964'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:19 user nova-compute[71317]: INFO os_vif [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:95:29:3f,bridge_name='br-int',has_traffic_filtering=True,id=de9378d0-c63f-48c3-9a25-e7d82b8c0f99,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapde9378d0-c6')
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] No VIF found with MAC fa:16:3e:95:29:3f, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.network.neutron [req-0a7f9130-472a-4acd-9c2a-35e8b0679851 req-5f22ed02-b7a8-4f5c-86e2-1bcee8552b38 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Updated VIF entry in instance network info cache for port de9378d0-c63f-48c3-9a25-e7d82b8c0f99. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG nova.network.neutron [req-0a7f9130-472a-4acd-9c2a-35e8b0679851 req-5f22ed02-b7a8-4f5c-86e2-1bcee8552b38 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Updating instance_info_cache with network_info: [{"id": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "address": "fa:16:3e:95:29:3f", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde9378d0-c6", "ovs_interfaceid": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0a7f9130-472a-4acd-9c2a-35e8b0679851 req-5f22ed02-b7a8-4f5c-86e2-1bcee8552b38 service nova] Releasing lock "refresh_cache-59e547de-a74b-4ccd-90de-e7b61cba6964" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:18:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-52ea88b2-8b6a-45ea-a06c-21ba91411c15 req-3ea586d9-74e9-48ad-a720-cade36b32932 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Received event network-vif-plugged-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-52ea88b2-8b6a-45ea-a06c-21ba91411c15 req-3ea586d9-74e9-48ad-a720-cade36b32932 service nova] Acquiring lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-52ea88b2-8b6a-45ea-a06c-21ba91411c15 req-3ea586d9-74e9-48ad-a720-cade36b32932 service nova] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-52ea88b2-8b6a-45ea-a06c-21ba91411c15 req-3ea586d9-74e9-48ad-a720-cade36b32932 service nova] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-52ea88b2-8b6a-45ea-a06c-21ba91411c15 req-3ea586d9-74e9-48ad-a720-cade36b32932 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] No waiting events found dispatching network-vif-plugged-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:20 user nova-compute[71317]: WARNING nova.compute.manager [req-52ea88b2-8b6a-45ea-a06c-21ba91411c15 req-3ea586d9-74e9-48ad-a720-cade36b32932 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Received unexpected event network-vif-plugged-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 for instance with vm_state building and task_state spawning.
Apr 14 04:18:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:21 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:18:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:18:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-9b532a44-20b7-4590-babe-684c49598a62" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:18:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-9b532a44-20b7-4590-babe-684c49598a62" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:18:21 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:18:21 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Updating instance_info_cache with network_info: [{"id": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "address": "fa:16:3e:47:9f:40", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc45f9c53-8d", "ovs_interfaceid": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-9b532a44-20b7-4590-babe-684c49598a62" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:18:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:18:21 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:18:21 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:18:21 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:18:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445902.6875992, 59e547de-a74b-4ccd-90de-e7b61cba6964 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:18:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] VM Resumed (Lifecycle Event)
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:18:22 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Instance spawned successfully.
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:18:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445902.6933239, 59e547de-a74b-4ccd-90de-e7b61cba6964 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:18:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] VM Started (Lifecycle Event)
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-ac04d7d5-5723-47d9-b51c-07b1eea2250b req-e73e43ea-42d9-46ac-a278-bbc3ceb8e527 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Received event network-vif-plugged-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ac04d7d5-5723-47d9-b51c-07b1eea2250b req-e73e43ea-42d9-46ac-a278-bbc3ceb8e527 service nova] Acquiring lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ac04d7d5-5723-47d9-b51c-07b1eea2250b req-e73e43ea-42d9-46ac-a278-bbc3ceb8e527 service nova] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ac04d7d5-5723-47d9-b51c-07b1eea2250b req-e73e43ea-42d9-46ac-a278-bbc3ceb8e527 service nova] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-ac04d7d5-5723-47d9-b51c-07b1eea2250b req-e73e43ea-42d9-46ac-a278-bbc3ceb8e527 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] No waiting events found dispatching network-vif-plugged-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:22 user nova-compute[71317]: WARNING nova.compute.manager [req-ac04d7d5-5723-47d9-b51c-07b1eea2250b req-e73e43ea-42d9-46ac-a278-bbc3ceb8e527 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Received unexpected event network-vif-plugged-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 for instance with vm_state building and task_state spawning.
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:18:22 user nova-compute[71317]: INFO nova.compute.manager [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Took 5.62 seconds to spawn the instance on the hypervisor.
Apr 14 04:18:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:18:22 user nova-compute[71317]: INFO nova.compute.manager [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Took 6.45 seconds to build instance.
Apr 14 04:18:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e906d53a-a008-41bb-816d-71c21c7af2de tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.551s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.217s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-bf26deef-253e-402e-967b-01cc86417aaf req-301f1ff6-3e6b-4998-9c57-453c7a75cb31 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-changed-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-bf26deef-253e-402e-967b-01cc86417aaf req-301f1ff6-3e6b-4998-9c57-453c7a75cb31 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Refreshing instance network info cache due to event network-changed-df63af46-27d1-4086-ae2c-39e8bc829bc7. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bf26deef-253e-402e-967b-01cc86417aaf req-301f1ff6-3e6b-4998-9c57-453c7a75cb31 service nova] Acquiring lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bf26deef-253e-402e-967b-01cc86417aaf req-301f1ff6-3e6b-4998-9c57-453c7a75cb31 service nova] Acquired lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG nova.network.neutron [req-bf26deef-253e-402e-967b-01cc86417aaf req-301f1ff6-3e6b-4998-9c57-453c7a75cb31 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Refreshing network info cache for port df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG nova.network.neutron [req-bf26deef-253e-402e-967b-01cc86417aaf req-301f1ff6-3e6b-4998-9c57-453c7a75cb31 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updated VIF entry in instance network info cache for port df63af46-27d1-4086-ae2c-39e8bc829bc7. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG nova.network.neutron [req-bf26deef-253e-402e-967b-01cc86417aaf req-301f1ff6-3e6b-4998-9c57-453c7a75cb31 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updating instance_info_cache with network_info: [{"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bf26deef-253e-402e-967b-01cc86417aaf req-301f1ff6-3e6b-4998-9c57-453c7a75cb31 service nova] Releasing lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.126s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vdc) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681445893.6463537, 0ff97860-2441-437c-b118-6089c3cc4160 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:18:28 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] VM Stopped (Lifecycle Event)
Apr 14 04:18:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:28 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:18:28 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7130MB free_disk=26.556598663330078GB free_vcpus=-1 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7b195d0e-0df0-49df-9661-4ac0b1c3fcf8 None None] [instance: 0ff97860-2441-437c-b118-6089c3cc4160] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.081s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 55ba1cae-2b2f-43de-86db-996a8791fb02 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance da3b4651-aaa0-430d-85b8-c15de386d08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 59e547de-a74b-4ccd-90de-e7b61cba6964 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 9b532a44-20b7-4590-babe-684c49598a62 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance e37e27b4-0813-4df9-ac66-1516334c12cb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance e5344b0d-2d73-40c7-8ac3-535396e985a4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 13 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:18:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=2176MB phys_disk=40GB used_disk=12GB total_vcpus=12 used_vcpus=13 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:29 user nova-compute[71317]: INFO nova.compute.manager [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Attaching volume b7e94002-3eeb-442d-b224-ef83dacfa5a4 to /dev/vdb
Apr 14 04:18:29 user nova-compute[71317]: DEBUG os_brick.utils [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4219493e-9a06-4046-a096-fa67e8f3ef2d]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6dad0a1a-67b5-4a7c-a499-7aa79b7fe604]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[55a24837-f309-4743-80e0-5a0ae2e10379]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4453fe2-cd87-4c23-a319-64ad62ee6a1c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:18:29 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[67f940a1-8923-49e7-b936-790e9df20508]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG os_brick.utils [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] <== get_connector_properties: return (169ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updating existing volume attachment record: f57596f4-3157-4f5b-b9aa-52470f8dbf1c {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:18:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.699s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:18:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:18:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "e37e27b4-0813-4df9-ac66-1516334c12cb" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:30 user nova-compute[71317]: DEBUG nova.objects.instance [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lazy-loading 'flavor' on Instance uuid e37e27b4-0813-4df9-ac66-1516334c12cb {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.180s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "e37e27b4-0813-4df9-ac66-1516334c12cb" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:31 user nova-compute[71317]: INFO nova.compute.manager [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Attaching volume 09a4582c-404d-4225-baf5-59b74e42325d to /dev/vdb
Apr 14 04:18:31 user nova-compute[71317]: DEBUG os_brick.utils [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e37161c3-c118-438a-97ea-3e17b7a807ea]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0483a5c9-e298-4534-8c5f-900f3c9abfb9]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1b453e0a-4dab-4008-aa5d-896b6a4b0bab]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b8320c7-dc6a-4087-ad37-18dfd8f88675]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:18:31 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:18:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[edee4452-5df4-494d-afbd-20ebb3885d2a]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG os_brick.utils [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] <== get_connector_properties: return (121ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:18:31 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Updating existing volume attachment record: 7ae45d61-2bc7-4ef5-8288-3215872da780 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:18:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mfOroQgL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:18:34 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:18:34 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:18:34 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:18:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[82d23a54-87cb-4a81-bf1a-21269811379c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:18:34 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[62c4c805-5e64-43f3-a51a-ce0ff8975bef]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4] added
Apr 14 04:18:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ddcc9cd9-2eba-4af0-9b42-bd6b33dae88a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[68b77016-e3fc-43bf-b5fe-43d32e0ae2bf]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mfOroQgL {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mfOroQgL" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1617c3fe-5be4-4759-9fa7-6c8eb37a55c7]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'mfOroQgL'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3ed5a56-4d5d-4aea-b8b8-abf24881c602]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1fc23375-07e5-4806-8afd-252a3ae04f65]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:34 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:34 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:18:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:34 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:34 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:18:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --login" returned: 0 in 0.071s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[532a4b98-63fa-4967-9c2a-3d7885019f8e]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260]
Apr 14 04:18:34 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260] successful.
Apr 14 04:18:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7a34950-e8aa-4961-96e9-aecc99e2cd7e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5dec2ae-c69b-45d5-af2f-ad84ccdcf199]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:34 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:34 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:18:34 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:18:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:34 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:34 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:18:34 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:18:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.336s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] HCTL ('36', '-', '-', 0) found on session 5 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.033s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b63850d3-144c-4d9a-b210-f932c8ab352a]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Searching for a device in session 5 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:18:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mfOroQgL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-77a54ad6-a303-4c44-bb6e-b81763dda202 req-54190368-4b61-4f19-bd10-5a7949900589 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-changed-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-77a54ad6-a303-4c44-bb6e-b81763dda202 req-54190368-4b61-4f19-bd10-5a7949900589 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Refreshing instance network info cache due to event network-changed-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-77a54ad6-a303-4c44-bb6e-b81763dda202 req-54190368-4b61-4f19-bd10-5a7949900589 service nova] Acquiring lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-77a54ad6-a303-4c44-bb6e-b81763dda202 req-54190368-4b61-4f19-bd10-5a7949900589 service nova] Acquired lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG nova.network.neutron [req-77a54ad6-a303-4c44-bb6e-b81763dda202 req-54190368-4b61-4f19-bd10-5a7949900589 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Refreshing network info cache for port 3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:18:35 user nova-compute[71317]: INFO nova.compute.manager [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Rescuing
Apr 14 04:18:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquired lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.399s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] <== connect_volume: return (1402ms) {'type': 'block', 'scsi_wwn': '23463366364646231', 'path': '/dev/sde'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23463366364646231', 'path': '/dev/sde'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:18:35 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:18:35 user nova-compute[71317]:   <source dev="/dev/sde"/>
Apr 14 04:18:35 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:18:35 user nova-compute[71317]:   <serial>b7e94002-3eeb-442d-b224-ef83dacfa5a4</serial>
Apr 14 04:18:35 user nova-compute[71317]: </disk>
Apr 14 04:18:35 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG nova.network.neutron [req-77a54ad6-a303-4c44-bb6e-b81763dda202 req-54190368-4b61-4f19-bd10-5a7949900589 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updated VIF entry in instance network info cache for port 3a3e3b13-7b4e-46c8-9123-5c1f121a38b9. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG nova.network.neutron [req-77a54ad6-a303-4c44-bb6e-b81763dda202 req-54190368-4b61-4f19-bd10-5a7949900589 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating instance_info_cache with network_info: [{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-77a54ad6-a303-4c44-bb6e-b81763dda202 req-54190368-4b61-4f19-bd10-5a7949900589 service nova] Releasing lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:18:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No VIF found with MAC fa:16:3e:68:ca:2f, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Updating instance_info_cache with network_info: [{"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Releasing lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f7c0f4a-1f73-4e30-9810-e25252fba8f7 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.103s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.compute.manager [req-1e6bb2e6-de8f-4768-8c5e-b4ec0045d035 req-6392b79d-b8b8-4c49-b3dc-fb5c4c262f4d service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received event network-vif-unplugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1e6bb2e6-de8f-4768-8c5e-b4ec0045d035 req-6392b79d-b8b8-4c49-b3dc-fb5c4c262f4d service nova] Acquiring lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1e6bb2e6-de8f-4768-8c5e-b4ec0045d035 req-6392b79d-b8b8-4c49-b3dc-fb5c4c262f4d service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1e6bb2e6-de8f-4768-8c5e-b4ec0045d035 req-6392b79d-b8b8-4c49-b3dc-fb5c4c262f4d service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.compute.manager [req-1e6bb2e6-de8f-4768-8c5e-b4ec0045d035 req-6392b79d-b8b8-4c49-b3dc-fb5c4c262f4d service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] No waiting events found dispatching network-vif-unplugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:36 user nova-compute[71317]: WARNING nova.compute.manager [req-1e6bb2e6-de8f-4768-8c5e-b4ec0045d035 req-6392b79d-b8b8-4c49-b3dc-fb5c4c262f4d service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received unexpected event network-vif-unplugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 for instance with vm_state active and task_state rescuing.
Apr 14 04:18:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'UAXqaniM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:18:36 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:18:36 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:18:36 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[439d6ee0-0021-43ca-8f73-2ffcd2e46b27]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:18:36 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[68186538-e8fb-448b-879c-2f50a5a7bb0c]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d] added
Apr 14 04:18:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7617378-466b-4da4-bbcc-8d854d5acffb]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2872b33d-9859-4fb1-b9f0-47ab40fad945]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v UAXqaniM {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v UAXqaniM" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7782bb4b-c224-4546-be0f-5eab7e695165]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'UAXqaniM'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb8aceda-53bb-45f5-a501-824f126b3902]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f839f5e-8612-4dbb-bed1-1806a99e9894]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:18:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:18:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Instance destroyed successfully.
Apr 14 04:18:36 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Attempting rescue
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=71317) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Instance directory exists: not creating {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 04:18:36 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Creating image(s)
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "/opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "/opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "/opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.objects.instance [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'trusted_certs' on Instance uuid 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --login" returned: 0 in 0.075s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7b870d1-b050-4a01-9ef5-b9e79b623421]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d, portal: 172.16.0.220,3260]
Apr 14 04:18:36 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d, portal: 172.16.0.220,3260] successful.
Apr 14 04:18:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6f5cc6dd-c92a-4bfd-b88a-4b60945f6333 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6f5cc6dd-c92a-4bfd-b88a-4b60945f6333 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" acquired by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.compute.manager [None req-6f5cc6dd-c92a-4bfd-b88a-4b60945f6333 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.compute.manager [None req-6f5cc6dd-c92a-4bfd-b88a-4b60945f6333 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Stopping instance; current vm_state: active, current task_state: powering-off, current DB power_state: 1, current VM power_state: 1 {{(pid=71317) do_stop_instance /opt/stack/nova/nova/compute/manager.py:3323}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.objects.instance [None req-6f5cc6dd-c92a-4bfd-b88a-4b60945f6333 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d2d189c-b336-4c60-bab9-a09115671f9c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c078252e-ab6b-46f4-a410-9e64aa0775eb]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d (non-flash)
Apr 14 04:18:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:18:36 user nova-compute[71317]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d (non-flash)
Apr 14 04:18:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.327s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] HCTL ('37', '-', '-', 0) found on session 6 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7666e478-f933-4508-afc1-af2af4a44a8f]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Searching for a device in session 6 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'UAXqaniM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.152s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue" returned: 0 in 0.056s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.211s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.objects.instance [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'migration_context' on Instance uuid 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Start _get_guest_xml network_info=[{"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "vif_mac": "fa:16:3e:f5:07:c5"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:18:36 user nova-compute[71317]: DEBUG nova.objects.instance [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'resources' on Instance uuid 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.objects.instance [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'numa_topology' on Instance uuid 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:37 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:18:37 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.objects.instance [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'vcpu_model' on Instance uuid 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1602894262',display_name='tempest-ServerRescueNegativeTestJSON-server-1602894262',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1602894262',id=7,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:00Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='33320dc2f2c14ecfa5abf6574029c36c',ramdisk_id='',reservation_id='r-5xk8owm0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-508041341',owner_user_name='tempest-ServerRescueNegativeTestJSON-508041341-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:01Z,user_data=None,user_id='5234914aec8e443b8a38ac6b38729031',uuid=2f7e20cd-33ef-4f83-bcdb-a13b9b51370a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "vif_mac": "fa:16:3e:f5:07:c5"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converting VIF {"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "vif_mac": "fa:16:3e:f5:07:c5"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f5:07:c5,bridge_name='br-int',has_traffic_filtering=True,id=1201ab18-d7ef-4a92-a77e-d0ab69858538,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1201ab18-d7') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.objects.instance [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'pci_devices' on Instance uuid 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:18:37 user nova-compute[71317]:   <uuid>2f7e20cd-33ef-4f83-bcdb-a13b9b51370a</uuid>
Apr 14 04:18:37 user nova-compute[71317]:   <name>instance-00000007</name>
Apr 14 04:18:37 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:18:37 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:18:37 user nova-compute[71317]:   <metadata>
Apr 14 04:18:37 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:18:37 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:18:37 user nova-compute[71317]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-1602894262</nova:name>
Apr 14 04:18:37 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:18:37</nova:creationTime>
Apr 14 04:18:37 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:18:37 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:18:37 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:18:37 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:18:37 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:18:37 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:18:37 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:18:37 user nova-compute[71317]:       <nova:owner>
Apr 14 04:18:37 user nova-compute[71317]:         <nova:user uuid="5234914aec8e443b8a38ac6b38729031">tempest-ServerRescueNegativeTestJSON-508041341-project-member</nova:user>
Apr 14 04:18:37 user nova-compute[71317]:         <nova:project uuid="33320dc2f2c14ecfa5abf6574029c36c">tempest-ServerRescueNegativeTestJSON-508041341</nova:project>
Apr 14 04:18:37 user nova-compute[71317]:       </nova:owner>
Apr 14 04:18:37 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:18:37 user nova-compute[71317]:       <nova:ports>
Apr 14 04:18:37 user nova-compute[71317]:         <nova:port uuid="1201ab18-d7ef-4a92-a77e-d0ab69858538">
Apr 14 04:18:37 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Apr 14 04:18:37 user nova-compute[71317]:         </nova:port>
Apr 14 04:18:37 user nova-compute[71317]:       </nova:ports>
Apr 14 04:18:37 user nova-compute[71317]:     </nova:instance>
Apr 14 04:18:37 user nova-compute[71317]:   </metadata>
Apr 14 04:18:37 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:18:37 user nova-compute[71317]:     <system>
Apr 14 04:18:37 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:18:37 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:18:37 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:18:37 user nova-compute[71317]:       <entry name="serial">2f7e20cd-33ef-4f83-bcdb-a13b9b51370a</entry>
Apr 14 04:18:37 user nova-compute[71317]:       <entry name="uuid">2f7e20cd-33ef-4f83-bcdb-a13b9b51370a</entry>
Apr 14 04:18:37 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:18:37 user nova-compute[71317]:     </system>
Apr 14 04:18:37 user nova-compute[71317]:   </sysinfo>
Apr 14 04:18:37 user nova-compute[71317]:   <os>
Apr 14 04:18:37 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:18:37 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:18:37 user nova-compute[71317]:   </os>
Apr 14 04:18:37 user nova-compute[71317]:   <features>
Apr 14 04:18:37 user nova-compute[71317]:     <acpi/>
Apr 14 04:18:37 user nova-compute[71317]:     <apic/>
Apr 14 04:18:37 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:18:37 user nova-compute[71317]:   </features>
Apr 14 04:18:37 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:18:37 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:18:37 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:18:37 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:18:37 user nova-compute[71317]:   </clock>
Apr 14 04:18:37 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:18:37 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:18:37 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:18:37 user nova-compute[71317]:   </cpu>
Apr 14 04:18:37 user nova-compute[71317]:   <devices>
Apr 14 04:18:37 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:18:37 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:18:37 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue"/>
Apr 14 04:18:37 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:18:37 user nova-compute[71317]:     </disk>
Apr 14 04:18:37 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:18:37 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:18:37 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk"/>
Apr 14 04:18:37 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:18:37 user nova-compute[71317]:     </disk>
Apr 14 04:18:37 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:18:37 user nova-compute[71317]:       <mac address="fa:16:3e:f5:07:c5"/>
Apr 14 04:18:37 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:18:37 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:18:37 user nova-compute[71317]:       <target dev="tap1201ab18-d7"/>
Apr 14 04:18:37 user nova-compute[71317]:     </interface>
Apr 14 04:18:37 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:18:37 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/console.log" append="off"/>
Apr 14 04:18:37 user nova-compute[71317]:     </serial>
Apr 14 04:18:37 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:18:37 user nova-compute[71317]:     <video>
Apr 14 04:18:37 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:18:37 user nova-compute[71317]:     </video>
Apr 14 04:18:37 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:18:37 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:18:37 user nova-compute[71317]:     </rng>
Apr 14 04:18:37 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:18:37 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:18:37 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:18:37 user nova-compute[71317]:     </memballoon>
Apr 14 04:18:37 user nova-compute[71317]:   </devices>
Apr 14 04:18:37 user nova-compute[71317]: </domain>
Apr 14 04:18:37 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:18:37 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Instance destroyed successfully.
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-35e400d6-5bdc-4a29-9f5d-af36f3665bac req-39232ac8-81ce-4364-9643-9362207bf065 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-unplugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-35e400d6-5bdc-4a29-9f5d-af36f3665bac req-39232ac8-81ce-4364-9643-9362207bf065 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-35e400d6-5bdc-4a29-9f5d-af36f3665bac req-39232ac8-81ce-4364-9643-9362207bf065 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-35e400d6-5bdc-4a29-9f5d-af36f3665bac req-39232ac8-81ce-4364-9643-9362207bf065 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-35e400d6-5bdc-4a29-9f5d-af36f3665bac req-39232ac8-81ce-4364-9643-9362207bf065 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-unplugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:37 user nova-compute[71317]: WARNING nova.compute.manager [req-35e400d6-5bdc-4a29-9f5d-af36f3665bac req-39232ac8-81ce-4364-9643-9362207bf065 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-unplugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state active and task_state powering-off.
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No VIF found with MAC fa:16:3e:f5:07:c5, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.360s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] <== connect_volume: return (1361ms) {'type': 'block', 'scsi_wwn': '26232393833383461', 'path': '/dev/sdf'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26232393833383461', 'path': '/dev/sdf'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.objects.instance [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lazy-loading 'flavor' on Instance uuid e37e27b4-0813-4df9-ac66-1516334c12cb {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:18:37 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:18:37 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:18:37 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:18:37 user nova-compute[71317]:   <serial>09a4582c-404d-4225-baf5-59b74e42325d</serial>
Apr 14 04:18:37 user nova-compute[71317]: </disk>
Apr 14 04:18:37 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:18:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:38 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance destroyed successfully.
Apr 14 04:18:38 user nova-compute[71317]: DEBUG nova.compute.manager [None req-6f5cc6dd-c92a-4bfd-b88a-4b60945f6333 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6f5cc6dd-c92a-4bfd-b88a-4b60945f6333 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" "released" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: held 1.490s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-0027fd09-e0fd-473b-820c-546442d0d342 req-bb0229d5-ca3c-4259-ba29-167b98956877 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received event network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0027fd09-e0fd-473b-820c-546442d0d342 req-bb0229d5-ca3c-4259-ba29-167b98956877 service nova] Acquiring lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0027fd09-e0fd-473b-820c-546442d0d342 req-bb0229d5-ca3c-4259-ba29-167b98956877 service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0027fd09-e0fd-473b-820c-546442d0d342 req-bb0229d5-ca3c-4259-ba29-167b98956877 service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-0027fd09-e0fd-473b-820c-546442d0d342 req-bb0229d5-ca3c-4259-ba29-167b98956877 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] No waiting events found dispatching network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:38 user nova-compute[71317]: WARNING nova.compute.manager [req-0027fd09-e0fd-473b-820c-546442d0d342 req-bb0229d5-ca3c-4259-ba29-167b98956877 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received unexpected event network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 for instance with vm_state active and task_state rescuing.
Apr 14 04:18:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] No VIF found with MAC fa:16:3e:61:5e:bb, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:18:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-02b363a5-ff13-4502-8b19-110d6817aab6 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.548s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "e37e27b4-0813-4df9-ac66-1516334c12cb" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:39 user nova-compute[71317]: INFO nova.compute.manager [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Terminating instance
Apr 14 04:18:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:39 user nova-compute[71317]: WARNING nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state stopped and task_state None.
Apr 14 04:18:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:39 user nova-compute[71317]: WARNING nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state stopped and task_state None.
Apr 14 04:18:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:39 user nova-compute[71317]: WARNING nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state stopped and task_state None.
Apr 14 04:18:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-unplugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-unplugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:39 user nova-compute[71317]: WARNING nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-unplugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state stopped and task_state None.
Apr 14 04:18:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:39 user nova-compute[71317]: WARNING nova.compute.manager [req-5f7881b9-65c3-4e3f-b0ef-1a5c13110339 req-75573994-a254-472f-b52b-f204c2166d79 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state stopped and task_state None.
Apr 14 04:18:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:39 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Instance destroyed successfully.
Apr 14 04:18:39 user nova-compute[71317]: DEBUG nova.objects.instance [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lazy-loading 'resources' on Instance uuid e37e27b4-0813-4df9-ac66-1516334c12cb {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-238847393',display_name='tempest-DeleteServersTestJSON-server-238847393',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-238847393',id=4,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:16:52Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ce5e634f187b470085e7bfb3c1cfcbe7',ramdisk_id='',reservation_id='r-2fxy9zb2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-DeleteServersTestJSON-2046031715',owner_user_name='tempest-DeleteServersTestJSON-2046031715-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:16:53Z,user_data=None,user_id='fab852bb302f44b1af63083c99ea244c',uuid=e37e27b4-0813-4df9-ac66-1516334c12cb,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2a342575-05e3-4179-a370-6d39ac4f815d", "address": "fa:16:3e:61:5e:bb", "network": {"id": "2067a33c-66ab-4b9b-a1d5-a8dbb603f119", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1353356979-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ce5e634f187b470085e7bfb3c1cfcbe7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2a342575-05", "ovs_interfaceid": "2a342575-05e3-4179-a370-6d39ac4f815d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Converting VIF {"id": "2a342575-05e3-4179-a370-6d39ac4f815d", "address": "fa:16:3e:61:5e:bb", "network": {"id": "2067a33c-66ab-4b9b-a1d5-a8dbb603f119", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1353356979-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ce5e634f187b470085e7bfb3c1cfcbe7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2a342575-05", "ovs_interfaceid": "2a342575-05e3-4179-a370-6d39ac4f815d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:61:5e:bb,bridge_name='br-int',has_traffic_filtering=True,id=2a342575-05e3-4179-a370-6d39ac4f815d,network=Network(2067a33c-66ab-4b9b-a1d5-a8dbb603f119),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2a342575-05') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_vif [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:61:5e:bb,bridge_name='br-int',has_traffic_filtering=True,id=2a342575-05e3-4179-a370-6d39ac4f815d,network=Network(2067a33c-66ab-4b9b-a1d5-a8dbb603f119),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2a342575-05') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2a342575-05, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:40 user nova-compute[71317]: INFO os_vif [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:61:5e:bb,bridge_name='br-int',has_traffic_filtering=True,id=2a342575-05e3-4179-a370-6d39ac4f815d,network=Network(2067a33c-66ab-4b9b-a1d5-a8dbb603f119),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2a342575-05')
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'UAXqaniM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:18:40 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d181e18d-f057-4b4b-a28d-9d687a4adcca]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:18:40 user nova-compute[71317]: No targets found.
Apr 14 04:18:40 user nova-compute[71317]: iSNS:
Apr 14 04:18:40 user nova-compute[71317]: No targets found.
Apr 14 04:18:40 user nova-compute[71317]: STATIC:
Apr 14 04:18:40 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282
Apr 14 04:18:40 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:18:40 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:18:40 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d
Apr 14 04:18:40 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:18:40 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:18:40 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b
Apr 14 04:18:40 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:18:40 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:18:40 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:18:40 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:18:40 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:18:40 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3
Apr 14 04:18:40 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:18:40 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:18:40 user nova-compute[71317]: FIRMWARE:
Apr 14 04:18:40 user nova-compute[71317]: No targets found.
Apr 14 04:18:40 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:18:40 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:18:40 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ced0ce8-950f-48d3-abd1-d2b0fa222087]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[94caf4ca-766d-41e8-b471-f8cf94194911]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:40 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:40 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:18:40 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:18:40 user nova-compute[71317]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d (non-flash)
Apr 14 04:18:40 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:18:40 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:18:40 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:18:40 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:18:40 user nova-compute[71317]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d (non-flash)
Apr 14 04:18:40 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d18abbe0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d'): ({'sdf'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Removing single pathed devices sdf {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[546a2464-d88b-44ae-8abc-34209508a4e5]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e882167d-88e3-4551-ae04-a7ac33f53db0]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Flushing IO for device /dev/sdf {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[41ef5114-7311-4bbe-95ab-c96a7d19561d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[976ed83a-6983-46a8-b70e-c353093a419e]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[46da3475-6e50-44a1-855c-d47fd5657418]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --logout" returned: 0 in 0.077s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d87bcaa-2595-4a6e-8dce-7ae5c915d480]: (4, ('Logging out of session [sid: 6, target: iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d, portal: 172.16.0.220,3260]\nLogout of [sid: 6, target: iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 6, target: iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d, portal: 172.16.0.220,3260]
Apr 14 04:18:40 user nova-compute[71317]: Logout of [sid: 6, target: iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d, portal: 172.16.0.220,3260] successful.
Apr 14 04:18:40 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:09a4582c-404d-4225-baf5-59b74e42325d -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fe65efe-bad2-48ee-b957-f31cbd197c54]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.348s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] <== disconnect_volume: return (351ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:18:40 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Deleting instance files /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb_del
Apr 14 04:18:40 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Deletion of /opt/stack/data/nova/instances/e37e27b4-0813-4df9-ac66-1516334c12cb_del complete
Apr 14 04:18:40 user nova-compute[71317]: INFO nova.compute.manager [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Took 1.46 seconds to destroy the instance on the hypervisor.
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Received event network-vif-unplugged-2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] Acquiring lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] No waiting events found dispatching network-vif-unplugged-2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Received event network-vif-unplugged-2a342575-05e3-4179-a370-6d39ac4f815d for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-changed-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Refreshing instance network info cache due to event network-changed-0c62956f-a2d8-4932-b48f-469eba99b8e6. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] Acquiring lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] Acquired lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:18:40 user nova-compute[71317]: DEBUG nova.network.neutron [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Refreshing network info cache for port 0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.network.neutron [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updated VIF entry in instance network info cache for port 0c62956f-a2d8-4932-b48f-469eba99b8e6. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.network.neutron [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updating instance_info_cache with network_info: [{"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] Releasing lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Received event network-vif-plugged-2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] Acquiring lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] No waiting events found dispatching network-vif-plugged-2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:41 user nova-compute[71317]: WARNING nova.compute.manager [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Received unexpected event network-vif-plugged-2a342575-05e3-4179-a370-6d39ac4f815d for instance with vm_state active and task_state deleting.
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received event network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] Acquiring lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] No waiting events found dispatching network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:41 user nova-compute[71317]: WARNING nova.compute.manager [req-5519f5f3-9f4a-4247-85ea-d9c4b41f00d3 req-5ea51441-f92c-43a5-8c13-5bce4ecfbaae service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received unexpected event network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 for instance with vm_state active and task_state rescuing.
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:41 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Took 0.70 seconds to deallocate network for instance.
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-e47cffa9-c8b4-4cb5-8cb1-f2a0c85cf494 req-d4eb6eaa-81f4-4352-8e4e-c335b8aa4d9c service nova] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Received event network-vif-deleted-2a342575-05e3-4179-a370-6d39ac4f815d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:41 user nova-compute[71317]: INFO nova.compute.manager [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] instance snapshotting
Apr 14 04:18:41 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Beginning live snapshot process
Apr 14 04:18:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json -f qcow2 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json -f qcow2" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json -f qcow2 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445921.8925545, 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:18:41 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] VM Resumed (Lifecycle Event)
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bb12431b-80aa-4a55-8082-6ca0da9194ea tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json -f qcow2" returned: 0 in 0.158s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:41 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 14 04:18:41 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445921.8978703, 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:18:41 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] VM Started (Lifecycle Event)
Apr 14 04:18:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp8wccaqo3/249ac01ab4874f50a11ea9c8553ed208.delta 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp8wccaqo3/249ac01ab4874f50a11ea9c8553ed208.delta 1073741824" returned: 0 in 0.048s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:42 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Quiescing instance not available: QEMU guest agent is not enabled.
Apr 14 04:18:42 user nova-compute[71317]: INFO nova.compute.manager [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Took 0.91 seconds to detach 1 volumes for instance.
Apr 14 04:18:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.438s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a07b89b-6e7a-43f1-b2fb-5eeddcaf58d2 req-c4ce8fbe-122c-4884-a5a1-804b5c665f2e service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received event network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a07b89b-6e7a-43f1-b2fb-5eeddcaf58d2 req-c4ce8fbe-122c-4884-a5a1-804b5c665f2e service nova] Acquiring lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a07b89b-6e7a-43f1-b2fb-5eeddcaf58d2 req-c4ce8fbe-122c-4884-a5a1-804b5c665f2e service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a07b89b-6e7a-43f1-b2fb-5eeddcaf58d2 req-c4ce8fbe-122c-4884-a5a1-804b5c665f2e service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a07b89b-6e7a-43f1-b2fb-5eeddcaf58d2 req-c4ce8fbe-122c-4884-a5a1-804b5c665f2e service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] No waiting events found dispatching network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:42 user nova-compute[71317]: WARNING nova.compute.manager [req-2a07b89b-6e7a-43f1-b2fb-5eeddcaf58d2 req-c4ce8fbe-122c-4884-a5a1-804b5c665f2e service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received unexpected event network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 for instance with vm_state rescued and task_state None.
Apr 14 04:18:42 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Deleted allocations for instance e37e27b4-0813-4df9-ac66-1516334c12cb
Apr 14 04:18:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-895a79f1-1033-4bb2-b419-a92316f9bcf4 tempest-DeleteServersTestJSON-2046031715 tempest-DeleteServersTestJSON-2046031715-project-member] Lock "e37e27b4-0813-4df9-ac66-1516334c12cb" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.741s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] COPY block job progress, current cursor: 0 final cursor: 43778048 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:18:43 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] COPY block job progress, current cursor: 43778048 final cursor: 43778048 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:18:43 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Skipping quiescing instance: QEMU guest agent is not enabled.
Apr 14 04:18:43 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:18:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp8wccaqo3/249ac01ab4874f50a11ea9c8553ed208.delta /opt/stack/data/nova/instances/snapshots/tmp8wccaqo3/249ac01ab4874f50a11ea9c8553ed208 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp8wccaqo3/249ac01ab4874f50a11ea9c8553ed208.delta /opt/stack/data/nova/instances/snapshots/tmp8wccaqo3/249ac01ab4874f50a11ea9c8553ed208" returned: 0 in 0.230s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:43 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Snapshot extracted, beginning image upload
Apr 14 04:18:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:45 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Snapshot image upload complete
Apr 14 04:18:45 user nova-compute[71317]: INFO nova.compute.manager [None req-639e55d6-044f-49ee-bcfb-b7e30e9e409e tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Took 4.46 seconds to snapshot the instance on the hypervisor.
Apr 14 04:18:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:53 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681445918.2312953, da3b4651-aaa0-430d-85b8-c15de386d08f => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:18:53 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] VM Stopped (Lifecycle Event)
Apr 14 04:18:53 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c27c04f6-f6d9-4d7d-8ef3-1b2c8f6df7de None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:53 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c27c04f6-f6d9-4d7d-8ef3-1b2c8f6df7de None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: stopped, current task_state: None, current DB power_state: 4, VM power_state: 4 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:18:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-40008615-5009-4c67-a4c0-b584ce046145 req-a8ad2d47-c68e-4739-a482-852b482f599f service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Received event network-changed-2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-40008615-5009-4c67-a4c0-b584ce046145 req-a8ad2d47-c68e-4739-a482-852b482f599f service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Refreshing instance network info cache due to event network-changed-2d24208b-94ce-4364-86a4-1758c24e9047. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:18:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-40008615-5009-4c67-a4c0-b584ce046145 req-a8ad2d47-c68e-4739-a482-852b482f599f service nova] Acquiring lock "refresh_cache-e5344b0d-2d73-40c7-8ac3-535396e985a4" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:18:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-40008615-5009-4c67-a4c0-b584ce046145 req-a8ad2d47-c68e-4739-a482-852b482f599f service nova] Acquired lock "refresh_cache-e5344b0d-2d73-40c7-8ac3-535396e985a4" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:18:53 user nova-compute[71317]: DEBUG nova.network.neutron [req-40008615-5009-4c67-a4c0-b584ce046145 req-a8ad2d47-c68e-4739-a482-852b482f599f service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Refreshing network info cache for port 2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:18:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-40008615-5009-4c67-a4c0-b584ce046145 req-a8ad2d47-c68e-4739-a482-852b482f599f service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Updated VIF entry in instance network info cache for port 2d24208b-94ce-4364-86a4-1758c24e9047. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:18:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-40008615-5009-4c67-a4c0-b584ce046145 req-a8ad2d47-c68e-4739-a482-852b482f599f service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Updating instance_info_cache with network_info: [{"id": "2d24208b-94ce-4364-86a4-1758c24e9047", "address": "fa:16:3e:22:c7:b3", "network": {"id": "69963625-cff2-4260-908f-82359d8b2f77", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-558617934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.176", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e0e636ba0ee9486f9f48905d886ab547", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d24208b-94", "ovs_interfaceid": "2d24208b-94ce-4364-86a4-1758c24e9047", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-40008615-5009-4c67-a4c0-b584ce046145 req-a8ad2d47-c68e-4739-a482-852b482f599f service nova] Releasing lock "refresh_cache-e5344b0d-2d73-40c7-8ac3-535396e985a4" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:18:54 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681445919.9911857, e37e27b4-0813-4df9-ac66-1516334c12cb => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:18:54 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] VM Stopped (Lifecycle Event)
Apr 14 04:18:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5851449c-c35b-4ad4-88c0-379728c0d6b4 None None] [instance: e37e27b4-0813-4df9-ac66-1516334c12cb] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:18:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-293a5fff-5f73-4e84-b67f-163bb04363bf req-2b12f65f-b4eb-4172-b040-340e6ebfb97f service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-changed-60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-293a5fff-5f73-4e84-b67f-163bb04363bf req-2b12f65f-b4eb-4172-b040-340e6ebfb97f service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Refreshing instance network info cache due to event network-changed-60579bc4-de13-48d7-b42d-2b9fc5597e45. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-293a5fff-5f73-4e84-b67f-163bb04363bf req-2b12f65f-b4eb-4172-b040-340e6ebfb97f service nova] Acquiring lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-293a5fff-5f73-4e84-b67f-163bb04363bf req-2b12f65f-b4eb-4172-b040-340e6ebfb97f service nova] Acquired lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-293a5fff-5f73-4e84-b67f-163bb04363bf req-2b12f65f-b4eb-4172-b040-340e6ebfb97f service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Refreshing network info cache for port 60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG nova.objects.instance [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lazy-loading 'flavor' on Instance uuid e5344b0d-2d73-40c7-8ac3-535396e985a4 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.058s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:56 user nova-compute[71317]: INFO nova.compute.manager [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Attaching volume 9c5e5b34-a82b-4011-88cb-968010ef59f9 to /dev/sdc
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "interface-8ed1dea8-e48a-49f7-924e-82cf639a1677-None" by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "interface-8ed1dea8-e48a-49f7-924e-82cf639a1677-None" acquired by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG nova.objects.instance [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lazy-loading 'flavor' on Instance uuid 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG os_brick.utils [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2d48ce8-5583-4725-8fbe-2cd6de313774]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[670427d7-774c-4e30-bbde-c668a9f96bab]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[29815474-1354-475c-a387-a0c506607749]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a304e56-b5a4-4685-928e-65fe3e598f1e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:18:56 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[524b3c5d-40b6-4453-8ee6-f84519f4c6a1]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG os_brick.utils [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] <== get_connector_properties: return (129ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Updating existing volume attachment record: 118396e2-4525-4f28-82cd-bb5ee6ba211a {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-293a5fff-5f73-4e84-b67f-163bb04363bf req-2b12f65f-b4eb-4172-b040-340e6ebfb97f service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Updated VIF entry in instance network info cache for port 60579bc4-de13-48d7-b42d-2b9fc5597e45. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-293a5fff-5f73-4e84-b67f-163bb04363bf req-2b12f65f-b4eb-4172-b040-340e6ebfb97f service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Updating instance_info_cache with network_info: [{"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.88", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-293a5fff-5f73-4e84-b67f-163bb04363bf req-2b12f65f-b4eb-4172-b040-340e6ebfb97f service nova] Releasing lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG nova.objects.instance [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lazy-loading 'pci_requests' on Instance uuid 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG nova.network.neutron [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:18:56 user nova-compute[71317]: DEBUG nova.policy [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0aeeeae7cbc340daa8cd81c1418794e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01cefa66350a4432ad71204b39e953be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:18:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:57 user nova-compute[71317]: DEBUG nova.network.neutron [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Successfully created port: 25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:18:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Successfully updated port: 25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:18:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:18:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquired lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:18:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:18:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-ae89b9bf-adb0-470f-9420-adb480035120 req-33a7756c-1154-412f-b122-f613dbc10a0c service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-changed-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-ae89b9bf-adb0-470f-9420-adb480035120 req-33a7756c-1154-412f-b122-f613dbc10a0c service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Refreshing instance network info cache due to event network-changed-25048f42-cfe5-48d1-8620-ac6ac6aa9b82. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:18:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ae89b9bf-adb0-470f-9420-adb480035120 req-33a7756c-1154-412f-b122-f613dbc10a0c service nova] Acquiring lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.network.neutron [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Updating instance_info_cache with network_info: [{"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.88", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Releasing lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ae89b9bf-adb0-470f-9420-adb480035120 req-33a7756c-1154-412f-b122-f613dbc10a0c service nova] Acquired lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-ae89b9bf-adb0-470f-9420-adb480035120 req-33a7756c-1154-412f-b122-f613dbc10a0c service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Refreshing network info cache for port 25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-420804238',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-420804238',id=10,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBvOf9hss+Yqylm6PnX6la1ymm/55r9GRsHJ0NsNe4bgFVG/3wZy3kBc49f/xLbrJ51dzLLHr/o6IQVXcYhlDKHgp4DCKWnVv/anFtE+/Na/vdZgAYvQkLcZYHFyD4dFRQ==',key_name='tempest-keypair-330344245',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='01cefa66350a4432ad71204b39e953be',ramdisk_id='',reservation_id='r-p91nk3xp',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-926384577',owner_user_name='tempest-TaggedAttachmentsTest-926384577-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:17:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='0aeeeae7cbc340daa8cd81c1418794e0',uuid=8ed1dea8-e48a-49f7-924e-82cf639a1677,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converting VIF {"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:14:36:0b,bridge_name='br-int',has_traffic_filtering=True,id=25048f42-cfe5-48d1-8620-ac6ac6aa9b82,network=Network(d28017dc-3976-4caa-a881-077e38a8b0c7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap25048f42-cf') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG os_vif [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:14:36:0b,bridge_name='br-int',has_traffic_filtering=True,id=25048f42-cfe5-48d1-8620-ac6ac6aa9b82,network=Network(d28017dc-3976-4caa-a881-077e38a8b0c7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap25048f42-cf') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap25048f42-cf, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap25048f42-cf, col_values=(('external_ids', {'iface-id': '25048f42-cfe5-48d1-8620-ac6ac6aa9b82', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:14:36:0b', 'vm-uuid': '8ed1dea8-e48a-49f7-924e-82cf639a1677'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:59 user nova-compute[71317]: INFO os_vif [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:14:36:0b,bridge_name='br-int',has_traffic_filtering=True,id=25048f42-cfe5-48d1-8620-ac6ac6aa9b82,network=Network(d28017dc-3976-4caa-a881-077e38a8b0c7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap25048f42-cf')
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-420804238',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-420804238',id=10,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBvOf9hss+Yqylm6PnX6la1ymm/55r9GRsHJ0NsNe4bgFVG/3wZy3kBc49f/xLbrJ51dzLLHr/o6IQVXcYhlDKHgp4DCKWnVv/anFtE+/Na/vdZgAYvQkLcZYHFyD4dFRQ==',key_name='tempest-keypair-330344245',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='01cefa66350a4432ad71204b39e953be',ramdisk_id='',reservation_id='r-p91nk3xp',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-926384577',owner_user_name='tempest-TaggedAttachmentsTest-926384577-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:17:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='0aeeeae7cbc340daa8cd81c1418794e0',uuid=8ed1dea8-e48a-49f7-924e-82cf639a1677,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converting VIF {"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:14:36:0b,bridge_name='br-int',has_traffic_filtering=True,id=25048f42-cfe5-48d1-8620-ac6ac6aa9b82,network=Network(d28017dc-3976-4caa-a881-077e38a8b0c7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap25048f42-cf') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] attach device xml: <interface type="ethernet">
Apr 14 04:18:59 user nova-compute[71317]:   <mac address="fa:16:3e:14:36:0b"/>
Apr 14 04:18:59 user nova-compute[71317]:   <model type="virtio"/>
Apr 14 04:18:59 user nova-compute[71317]:   <mtu size="1442"/>
Apr 14 04:18:59 user nova-compute[71317]:   <target dev="tap25048f42-cf"/>
Apr 14 04:18:59 user nova-compute[71317]: </interface>
Apr 14 04:18:59 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-2ef0f6d7-5fc5-4077-ad6a-09a142a5a34f req-6f5e126a-acaf-4395-97d2-97045cc95a9b service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-vif-plugged-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2ef0f6d7-5fc5-4077-ad6a-09a142a5a34f req-6f5e126a-acaf-4395-97d2-97045cc95a9b service nova] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2ef0f6d7-5fc5-4077-ad6a-09a142a5a34f req-6f5e126a-acaf-4395-97d2-97045cc95a9b service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2ef0f6d7-5fc5-4077-ad6a-09a142a5a34f req-6f5e126a-acaf-4395-97d2-97045cc95a9b service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-2ef0f6d7-5fc5-4077-ad6a-09a142a5a34f req-6f5e126a-acaf-4395-97d2-97045cc95a9b service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] No waiting events found dispatching network-vif-plugged-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:18:59 user nova-compute[71317]: WARNING nova.compute.manager [req-2ef0f6d7-5fc5-4077-ad6a-09a142a5a34f req-6f5e126a-acaf-4395-97d2-97045cc95a9b service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received unexpected event network-vif-plugged-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 for instance with vm_state active and task_state None.
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] No BDM found with device name hda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] No VIF found with MAC fa:16:3e:58:93:64, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:18:59 user nova-compute[71317]:   <nova:package version="0.0.0"/>
Apr 14 04:18:59 user nova-compute[71317]:   <nova:name>tempest-device-tagging-server-420804238</nova:name>
Apr 14 04:18:59 user nova-compute[71317]:   <nova:creationTime>2023-04-14 04:18:59</nova:creationTime>
Apr 14 04:18:59 user nova-compute[71317]:   <nova:flavor name="m1.nano">
Apr 14 04:18:59 user nova-compute[71317]:     <nova:memory>128</nova:memory>
Apr 14 04:18:59 user nova-compute[71317]:     <nova:disk>1</nova:disk>
Apr 14 04:18:59 user nova-compute[71317]:     <nova:swap>0</nova:swap>
Apr 14 04:18:59 user nova-compute[71317]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:18:59 user nova-compute[71317]:     <nova:vcpus>1</nova:vcpus>
Apr 14 04:18:59 user nova-compute[71317]:   </nova:flavor>
Apr 14 04:18:59 user nova-compute[71317]:   <nova:owner>
Apr 14 04:18:59 user nova-compute[71317]:     <nova:user uuid="0aeeeae7cbc340daa8cd81c1418794e0">tempest-TaggedAttachmentsTest-926384577-project-member</nova:user>
Apr 14 04:18:59 user nova-compute[71317]:     <nova:project uuid="01cefa66350a4432ad71204b39e953be">tempest-TaggedAttachmentsTest-926384577</nova:project>
Apr 14 04:18:59 user nova-compute[71317]:   </nova:owner>
Apr 14 04:18:59 user nova-compute[71317]:   <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:18:59 user nova-compute[71317]:   <nova:ports>
Apr 14 04:18:59 user nova-compute[71317]:     <nova:port uuid="60579bc4-de13-48d7-b42d-2b9fc5597e45">
Apr 14 04:18:59 user nova-compute[71317]:       <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 04:18:59 user nova-compute[71317]:     </nova:port>
Apr 14 04:18:59 user nova-compute[71317]:     <nova:port uuid="25048f42-cfe5-48d1-8620-ac6ac6aa9b82">
Apr 14 04:18:59 user nova-compute[71317]:       <nova:ip type="fixed" address="10.10.10.130" ipVersion="4"/>
Apr 14 04:18:59 user nova-compute[71317]:     </nova:port>
Apr 14 04:18:59 user nova-compute[71317]:   </nova:ports>
Apr 14 04:18:59 user nova-compute[71317]: </nova:instance>
Apr 14 04:18:59 user nova-compute[71317]:  {{(pid=71317) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-ae89b9bf-adb0-470f-9420-adb480035120 req-33a7756c-1154-412f-b122-f613dbc10a0c service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Updated VIF entry in instance network info cache for port 25048f42-cfe5-48d1-8620-ac6ac6aa9b82. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-ae89b9bf-adb0-470f-9420-adb480035120 req-33a7756c-1154-412f-b122-f613dbc10a0c service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Updating instance_info_cache with network_info: [{"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.88", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6ef74e9c-4d36-4200-be89-d939e4e68280 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "interface-8ed1dea8-e48a-49f7-924e-82cf639a1677-None" "released" by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" :: held 3.240s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ae89b9bf-adb0-470f-9420-adb480035120 req-33a7756c-1154-412f-b122-f613dbc10a0c service nova] Releasing lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:18:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lazy-loading 'flavor' on Instance uuid 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.054s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:00 user nova-compute[71317]: INFO nova.compute.manager [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Attaching volume 80a9f7e6-e364-4c2b-9814-96061ad47dd6 to /dev/vdb
Apr 14 04:19:00 user nova-compute[71317]: DEBUG os_brick.utils [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7f830d0-7766-40c1-96f1-cbc2bb3f609e]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[201a323d-89f8-4650-9327-e99e91f125c1]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9fb5942a-6faf-4564-b198-257299563c33]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5462c55b-37d4-4717-9712-abe1e0047e26]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:19:00 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:19:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f0a3099-7075-418f-96f8-566ab17c9a56]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG os_brick.utils [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] <== get_connector_properties: return (117ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:00 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Updating existing volume attachment record: 5113dbff-bf3f-49bc-87bc-94ce948ef193 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SLYEVqyw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:01 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:01 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:01 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[da13af83-ab5b-40d6-9d5a-58c2cfa3238e]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:19:01 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c295fc2f-0b5d-4c1a-b09f-eb6436843907]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9] added
Apr 14 04:19:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[da8d28b0-e9b3-4e4c-8926-d4745135a610]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a8852c1-ef66-427b-abf6-9c232318bfc4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SLYEVqyw {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SLYEVqyw" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee7ae89a-a5ec-4f3d-a704-944777701928]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'SLYEVqyw'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b70b6aa8-a53c-443e-9831-2c08ee2f7ddc]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4181c98-f330-4f26-927c-c611a349b92e]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --login" returned: 0 in 0.071s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1760c9f9-4e58-4ae1-8b29-7c86f7127238]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9, portal: 172.16.0.220,3260]
Apr 14 04:19:01 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[571317aa-08a6-450b-8289-86ea1c326402]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[94695bd9-5cbb-4567-b0a0-c0178a4abff7]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 (non-flash)
Apr 14 04:19:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.306s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] HCTL ('37', '-', '-', 0) found on session 7 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[59187994-37a1-4c79-9901-4867a51145f2]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Searching for a device in session 7 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SLYEVqyw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-0ff43957-e670-46ca-88ec-0313ce5d9358 req-d01d3625-e551-4bef-acb2-e0b89b2240d1 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-vif-plugged-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0ff43957-e670-46ca-88ec-0313ce5d9358 req-d01d3625-e551-4bef-acb2-e0b89b2240d1 service nova] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0ff43957-e670-46ca-88ec-0313ce5d9358 req-d01d3625-e551-4bef-acb2-e0b89b2240d1 service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0ff43957-e670-46ca-88ec-0313ce5d9358 req-d01d3625-e551-4bef-acb2-e0b89b2240d1 service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-0ff43957-e670-46ca-88ec-0313ce5d9358 req-d01d3625-e551-4bef-acb2-e0b89b2240d1 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] No waiting events found dispatching network-vif-plugged-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:01 user nova-compute[71317]: WARNING nova.compute.manager [req-0ff43957-e670-46ca-88ec-0313ce5d9358 req-d01d3625-e551-4bef-acb2-e0b89b2240d1 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received unexpected event network-vif-plugged-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 for instance with vm_state active and task_state None.
Apr 14 04:19:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.345s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] <== connect_volume: return (1347ms) {'type': 'block', 'scsi_wwn': '23162393337373236', 'path': '/dev/sdf'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23162393337373236', 'path': '/dev/sdf'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:19:02 user nova-compute[71317]: DEBUG nova.objects.instance [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lazy-loading 'flavor' on Instance uuid e5344b0d-2d73-40c7-8ac3-535396e985a4 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:19:02 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:19:02 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:19:02 user nova-compute[71317]:   <target dev="sdc" bus="scsi"/>
Apr 14 04:19:02 user nova-compute[71317]:   <serial>9c5e5b34-a82b-4011-88cb-968010ef59f9</serial>
Apr 14 04:19:02 user nova-compute[71317]:   <address type="drive" controller="0" unit="2"/>
Apr 14 04:19:02 user nova-compute[71317]: </disk>
Apr 14 04:19:02 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] No BDM found with device name sda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] No BDM found with device name sdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] No BDM found with device name sdc, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] No VIF found with MAC fa:16:3e:22:c7:b3, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3cc4c0e3-9d88-4f9b-bf43-f53e51b73186 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.762s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG nova.objects.instance [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lazy-loading 'flavor' on Instance uuid b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.052s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:03 user nova-compute[71317]: INFO nova.compute.manager [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Attaching volume f3255f25-0334-46dd-a514-12a56a1deffd to /dev/vdb
Apr 14 04:19:03 user nova-compute[71317]: DEBUG os_brick.utils [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7628b9af-4e1f-496a-a6c8-64f5e4e48cf4]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3cbceed-7529-4473-8582-b3bdf7409d95]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c254cfd4-b0a6-487b-b207-e7840eed1119]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[047dae4b-e409-4ae8-be4c-e2058c8739fd]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:19:03 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcf3a4bf-bf50-4104-ad98-ce1e51698524]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG os_brick.utils [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] <== get_connector_properties: return (119ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Updating existing volume attachment record: 92fba555-68e3-43bf-963d-49e70696a603 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG nova.compute.manager [req-5c384eb1-00da-4da4-b8bb-2b0b67c68cf5 req-a023f1ff-4593-4ad2-b479-9412f1f66c63 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Received event network-changed-3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG nova.compute.manager [req-5c384eb1-00da-4da4-b8bb-2b0b67c68cf5 req-a023f1ff-4593-4ad2-b479-9412f1f66c63 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Refreshing instance network info cache due to event network-changed-3f178241-75e2-4e0a-8d92-e647d05ed85e. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5c384eb1-00da-4da4-b8bb-2b0b67c68cf5 req-a023f1ff-4593-4ad2-b479-9412f1f66c63 service nova] Acquiring lock "refresh_cache-b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5c384eb1-00da-4da4-b8bb-2b0b67c68cf5 req-a023f1ff-4593-4ad2-b479-9412f1f66c63 service nova] Acquired lock "refresh_cache-b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:19:03 user nova-compute[71317]: DEBUG nova.network.neutron [req-5c384eb1-00da-4da4-b8bb-2b0b67c68cf5 req-a023f1ff-4593-4ad2-b479-9412f1f66c63 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Refreshing network info cache for port 3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "313835fa-6405-41ba-bc89-c4bfc119326a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "313835fa-6405-41ba-bc89-c4bfc119326a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.compute.manager [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:19:04 user nova-compute[71317]: INFO nova.compute.claims [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Claim successful on node user
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.network.neutron [req-5c384eb1-00da-4da4-b8bb-2b0b67c68cf5 req-a023f1ff-4593-4ad2-b479-9412f1f66c63 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Updated VIF entry in instance network info cache for port 3f178241-75e2-4e0a-8d92-e647d05ed85e. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.network.neutron [req-5c384eb1-00da-4da4-b8bb-2b0b67c68cf5 req-a023f1ff-4593-4ad2-b479-9412f1f66c63 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Updating instance_info_cache with network_info: [{"id": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "address": "fa:16:3e:cd:d0:ea", "network": {"id": "43bde039-802b-47d6-9194-3fcc8d89149f", "bridge": "br-int", "label": "tempest-TestVolumeSwap-57267579-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.78", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ffadee5eeb7f4b42ae93e2e2578ef897", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3f178241-75", "ovs_interfaceid": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5c384eb1-00da-4da4-b8bb-2b0b67c68cf5 req-a023f1ff-4593-4ad2-b479-9412f1f66c63 service nova] Releasing lock "refresh_cache-b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:04 user nova-compute[71317]: INFO nova.compute.manager [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Detaching volume 9c5e5b34-a82b-4011-88cb-968010ef59f9
Apr 14 04:19:04 user nova-compute[71317]: INFO nova.virt.block_device [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Attempting to driver detach volume 9c5e5b34-a82b-4011-88cb-968010ef59f9 from mountpoint /dev/sdc
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Attempting to detach device sdc from instance e5344b0d-2d73-40c7-8ac3-535396e985a4 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:19:04 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:19:04 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:19:04 user nova-compute[71317]:   <target dev="sdc" bus="scsi"/>
Apr 14 04:19:04 user nova-compute[71317]:   <serial>9c5e5b34-a82b-4011-88cb-968010ef59f9</serial>
Apr 14 04:19:04 user nova-compute[71317]:   <address type="drive" controller="0" bus="0" target="0" unit="2"/>
Apr 14 04:19:04 user nova-compute[71317]: </disk>
Apr 14 04:19:04 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:19:04 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Successfully detached device sdc from instance e5344b0d-2d73-40c7-8ac3-535396e985a4 from the persistent domain config.
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] (1/8): Attempting to detach device sdc with device alias scsi0-0-0-2 from instance e5344b0d-2d73-40c7-8ac3-535396e985a4 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:19:04 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:19:04 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:19:04 user nova-compute[71317]:   <target dev="sdc" bus="scsi"/>
Apr 14 04:19:04 user nova-compute[71317]:   <serial>9c5e5b34-a82b-4011-88cb-968010ef59f9</serial>
Apr 14 04:19:04 user nova-compute[71317]:   <address type="drive" controller="0" bus="0" target="0" unit="2"/>
Apr 14 04:19:04 user nova-compute[71317]: </disk>
Apr 14 04:19:04 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.499s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.compute.manager [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.compute.manager [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.network.neutron [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:19:04 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.compute.manager [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:19:04 user nova-compute[71317]: DEBUG nova.policy [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '6e422970cd8340bf996a135088094f51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '280aa55745a749b7baa134334d25c9c3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG nova.compute.manager [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:19:05 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Creating image(s)
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "/opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "/opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "/opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NtrRMlgi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:05 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:05 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:05 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a790f54-f01c-4034-b29b-0540efa2cea0]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:19:05 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681445945.1054938, e5344b0d-2d73-40c7-8ac3-535396e985a4 => scsi0-0-0-2> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Start waiting for the detach event from libvirt for device sdc with device alias scsi0-0-0-2 for instance e5344b0d-2d73-40c7-8ac3-535396e985a4 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:19:05 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Successfully detached device sdc from instance e5344b0d-2d73-40c7-8ac3-535396e985a4 from the live domain config.
Apr 14 04:19:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SLYEVqyw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:05 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[43df08fa-91a3-4e01-9547-932212a62ef6]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6] added
Apr 14 04:19:05 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[15ff3dc6-2885-442e-9cde-3f04bebab480]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[300288e5-66f6-4ed6-a951-76c1d104dfeb]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v NtrRMlgi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v NtrRMlgi" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b4557f6c-840b-4180-b897-16ba0ebb8e0c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'NtrRMlgi'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ef6251f-386a-4a39-ba07-6752691ff179]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a303db5-6c81-441c-a8af-e69f01dca5ed]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk 1073741824" returned: 0 in 0.048s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.184s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --login" returned: 0 in 0.076s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb33f487-59fa-4fdd-91a9-3bae1b41b08e]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6, portal: 172.16.0.220,3260]
Apr 14 04:19:05 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:05 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4172009-4105-4a7e-939d-8d6d3cbb8197]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[08c92183-e3bb-4fda-893a-98f4e5fc1424]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)
Apr 14 04:19:05 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.357s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] HCTL ('38', '-', '-', 0) found on session 8 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[55634624-e62c-4a1d-98bf-f6e8050c6675]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Searching for a device in session 8 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Connected to sdg using {'target_iqn': 'iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NtrRMlgi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.164s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Checking if we can resize image /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG nova.network.neutron [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Successfully created port: 243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Cannot resize image /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG nova.objects.instance [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lazy-loading 'migration_context' on Instance uuid 313835fa-6405-41ba-bc89-c4bfc119326a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Ensure instance console log exists: /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.network.neutron [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Successfully updated port: 243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "refresh_cache-313835fa-6405-41ba-bc89-c4bfc119326a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquired lock "refresh_cache-313835fa-6405-41ba-bc89-c4bfc119326a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.network.neutron [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-42c0f7f2-143e-4596-b8f7-33d7d46c1d50 req-b2ea00c2-bf7b-495e-8bec-04e1eff57ffd service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Received event network-changed-243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-42c0f7f2-143e-4596-b8f7-33d7d46c1d50 req-b2ea00c2-bf7b-495e-8bec-04e1eff57ffd service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Refreshing instance network info cache due to event network-changed-243334e0-8471-4569-bb6a-38c530471bb4. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-42c0f7f2-143e-4596-b8f7-33d7d46c1d50 req-b2ea00c2-bf7b-495e-8bec-04e1eff57ffd service nova] Acquiring lock "refresh_cache-313835fa-6405-41ba-bc89-c4bfc119326a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.network.neutron [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.400s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] <== connect_volume: return (1403ms) {'type': 'block', 'scsi_wwn': '23463366236626563', 'path': '/dev/sdg'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23463366236626563', 'path': '/dev/sdg'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 1.353s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lazy-loading 'flavor' on Instance uuid 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b6b6fac-6ea3-494f-80a9-af059ac66f84]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:19:06 user nova-compute[71317]: No targets found.
Apr 14 04:19:06 user nova-compute[71317]: iSNS:
Apr 14 04:19:06 user nova-compute[71317]: No targets found.
Apr 14 04:19:06 user nova-compute[71317]: STATIC:
Apr 14 04:19:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282
Apr 14 04:19:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6
Apr 14 04:19:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9
Apr 14 04:19:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b
Apr 14 04:19:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:19:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3
Apr 14 04:19:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:06 user nova-compute[71317]: FIRMWARE:
Apr 14 04:19:06 user nova-compute[71317]: No targets found.
Apr 14 04:19:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:19:06 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:19:06 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:19:06 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:19:06 user nova-compute[71317]:   <source dev="/dev/sdg"/>
Apr 14 04:19:06 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:19:06 user nova-compute[71317]:   <serial>80a9f7e6-e364-4c2b-9814-96061ad47dd6</serial>
Apr 14 04:19:06 user nova-compute[71317]: </disk>
Apr 14 04:19:06 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4a2c251-bb73-4e01-af10-4ffe20877f93]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7443514b-8d15-4e8d-bcd2-d98fbd479b5e]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:06 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:06 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:06 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:06 user nova-compute[71317]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 (non-flash)
Apr 14 04:19:06 user nova-compute[71317]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)
Apr 14 04:19:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:06 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:06 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:06 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:06 user nova-compute[71317]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 (non-flash)
Apr 14 04:19:06 user nova-compute[71317]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)
Apr 14 04:19:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84b42176d0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9'): ({'sdf'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Removing single pathed devices sdf {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[33a2b96a-f913-471f-a736-0be69f346b7e]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f34da3b8-9130-4ccf-a646-1abc7128a355]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Flushing IO for device /dev/sdf {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c6ae8b4-22c9-43c4-9950-a2e39a5ce583]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.047s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbeb1847-0fe5-4c75-b587-08a9d5bcee69]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0059c5e5-b813-4d23-bccd-95b27d89cd2c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.network.neutron [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Updating instance_info_cache with network_info: [{"id": "243334e0-8471-4569-bb6a-38c530471bb4", "address": "fa:16:3e:59:a5:ef", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap243334e0-84", "ovs_interfaceid": "243334e0-8471-4569-bb6a-38c530471bb4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --logout" returned: 0 in 0.061s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8aa8a72-bd01-4c62-9a10-5c49d82f9165]: (4, ('Logging out of session [sid: 7, target: iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9, portal: 172.16.0.220,3260]\nLogout of [sid: 7, target: iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 7, target: iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9, portal: 172.16.0.220,3260]
Apr 14 04:19:06 user nova-compute[71317]: Logout of [sid: 7, target: iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Releasing lock "refresh_cache-313835fa-6405-41ba-bc89-c4bfc119326a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Instance network_info: |[{"id": "243334e0-8471-4569-bb6a-38c530471bb4", "address": "fa:16:3e:59:a5:ef", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap243334e0-84", "ovs_interfaceid": "243334e0-8471-4569-bb6a-38c530471bb4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-42c0f7f2-143e-4596-b8f7-33d7d46c1d50 req-b2ea00c2-bf7b-495e-8bec-04e1eff57ffd service nova] Acquired lock "refresh_cache-313835fa-6405-41ba-bc89-c4bfc119326a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.network.neutron [req-42c0f7f2-143e-4596-b8f7-33d7d46c1d50 req-b2ea00c2-bf7b-495e-8bec-04e1eff57ffd service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Refreshing network info cache for port 243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Start _get_guest_xml network_info=[{"id": "243334e0-8471-4569-bb6a-38c530471bb4", "address": "fa:16:3e:59:a5:ef", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap243334e0-84", "ovs_interfaceid": "243334e0-8471-4569-bb6a-38c530471bb4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:19:06 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:19:06 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:19:03Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-667634161',display_name='tempest-VolumesActionsTest-instance-667634161',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-667634161',id=15,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='280aa55745a749b7baa134334d25c9c3',ramdisk_id='',reservation_id='r-1cpx0cta',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-447148503',owner_user_name='tempest-VolumesActionsTest-447148503-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:19:05Z,user_data=None,user_id='6e422970cd8340bf996a135088094f51',uuid=313835fa-6405-41ba-bc89-c4bfc119326a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "243334e0-8471-4569-bb6a-38c530471bb4", "address": "fa:16:3e:59:a5:ef", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap243334e0-84", "ovs_interfaceid": "243334e0-8471-4569-bb6a-38c530471bb4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Converting VIF {"id": "243334e0-8471-4569-bb6a-38c530471bb4", "address": "fa:16:3e:59:a5:ef", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap243334e0-84", "ovs_interfaceid": "243334e0-8471-4569-bb6a-38c530471bb4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:59:a5:ef,bridge_name='br-int',has_traffic_filtering=True,id=243334e0-8471-4569-bb6a-38c530471bb4,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap243334e0-84') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.objects.instance [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lazy-loading 'pci_devices' on Instance uuid 313835fa-6405-41ba-bc89-c4bfc119326a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c5e5b34-a82b-4011-88cb-968010ef59f9 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.033s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c374086c-97e3-40c1-8208-08f4f61eecce]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.350s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] <== disconnect_volume: return (1703ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:19:06 user nova-compute[71317]:   <uuid>313835fa-6405-41ba-bc89-c4bfc119326a</uuid>
Apr 14 04:19:06 user nova-compute[71317]:   <name>instance-0000000f</name>
Apr 14 04:19:06 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:19:06 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:19:06 user nova-compute[71317]:   <metadata>
Apr 14 04:19:06 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:19:06 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:19:06 user nova-compute[71317]:       <nova:name>tempest-VolumesActionsTest-instance-667634161</nova:name>
Apr 14 04:19:06 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:19:06</nova:creationTime>
Apr 14 04:19:06 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:19:06 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:19:06 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:19:06 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:19:06 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:19:06 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:19:06 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:19:06 user nova-compute[71317]:       <nova:owner>
Apr 14 04:19:06 user nova-compute[71317]:         <nova:user uuid="6e422970cd8340bf996a135088094f51">tempest-VolumesActionsTest-447148503-project-member</nova:user>
Apr 14 04:19:06 user nova-compute[71317]:         <nova:project uuid="280aa55745a749b7baa134334d25c9c3">tempest-VolumesActionsTest-447148503</nova:project>
Apr 14 04:19:06 user nova-compute[71317]:       </nova:owner>
Apr 14 04:19:06 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:19:06 user nova-compute[71317]:       <nova:ports>
Apr 14 04:19:06 user nova-compute[71317]:         <nova:port uuid="243334e0-8471-4569-bb6a-38c530471bb4">
Apr 14 04:19:06 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 14 04:19:06 user nova-compute[71317]:         </nova:port>
Apr 14 04:19:06 user nova-compute[71317]:       </nova:ports>
Apr 14 04:19:06 user nova-compute[71317]:     </nova:instance>
Apr 14 04:19:06 user nova-compute[71317]:   </metadata>
Apr 14 04:19:06 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:19:06 user nova-compute[71317]:     <system>
Apr 14 04:19:06 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:19:06 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:19:06 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:19:06 user nova-compute[71317]:       <entry name="serial">313835fa-6405-41ba-bc89-c4bfc119326a</entry>
Apr 14 04:19:06 user nova-compute[71317]:       <entry name="uuid">313835fa-6405-41ba-bc89-c4bfc119326a</entry>
Apr 14 04:19:06 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:19:06 user nova-compute[71317]:     </system>
Apr 14 04:19:06 user nova-compute[71317]:   </sysinfo>
Apr 14 04:19:06 user nova-compute[71317]:   <os>
Apr 14 04:19:06 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:19:06 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:19:06 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:19:06 user nova-compute[71317]:   </os>
Apr 14 04:19:06 user nova-compute[71317]:   <features>
Apr 14 04:19:06 user nova-compute[71317]:     <acpi/>
Apr 14 04:19:06 user nova-compute[71317]:     <apic/>
Apr 14 04:19:06 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:19:06 user nova-compute[71317]:   </features>
Apr 14 04:19:06 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:19:06 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:19:06 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:19:06 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:19:06 user nova-compute[71317]:   </clock>
Apr 14 04:19:06 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:19:06 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:19:06 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:19:06 user nova-compute[71317]:   </cpu>
Apr 14 04:19:06 user nova-compute[71317]:   <devices>
Apr 14 04:19:06 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:19:06 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:19:06 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk"/>
Apr 14 04:19:06 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:19:06 user nova-compute[71317]:     </disk>
Apr 14 04:19:06 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:19:06 user nova-compute[71317]:       <mac address="fa:16:3e:59:a5:ef"/>
Apr 14 04:19:06 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:19:06 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:19:06 user nova-compute[71317]:       <target dev="tap243334e0-84"/>
Apr 14 04:19:06 user nova-compute[71317]:     </interface>
Apr 14 04:19:06 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:19:06 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/console.log" append="off"/>
Apr 14 04:19:06 user nova-compute[71317]:     </serial>
Apr 14 04:19:06 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:19:06 user nova-compute[71317]:     <video>
Apr 14 04:19:06 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:19:06 user nova-compute[71317]:     </video>
Apr 14 04:19:06 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:19:06 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:19:06 user nova-compute[71317]:     </rng>
Apr 14 04:19:06 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:19:06 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:19:06 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:19:06 user nova-compute[71317]:     </memballoon>
Apr 14 04:19:06 user nova-compute[71317]:   </devices>
Apr 14 04:19:06 user nova-compute[71317]: </domain>
Apr 14 04:19:06 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:19:03Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-667634161',display_name='tempest-VolumesActionsTest-instance-667634161',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-667634161',id=15,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='280aa55745a749b7baa134334d25c9c3',ramdisk_id='',reservation_id='r-1cpx0cta',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-447148503',owner_user_name='tempest-VolumesActionsTest-447148503-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:19:05Z,user_data=None,user_id='6e422970cd8340bf996a135088094f51',uuid=313835fa-6405-41ba-bc89-c4bfc119326a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "243334e0-8471-4569-bb6a-38c530471bb4", "address": "fa:16:3e:59:a5:ef", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap243334e0-84", "ovs_interfaceid": "243334e0-8471-4569-bb6a-38c530471bb4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Converting VIF {"id": "243334e0-8471-4569-bb6a-38c530471bb4", "address": "fa:16:3e:59:a5:ef", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap243334e0-84", "ovs_interfaceid": "243334e0-8471-4569-bb6a-38c530471bb4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:59:a5:ef,bridge_name='br-int',has_traffic_filtering=True,id=243334e0-8471-4569-bb6a-38c530471bb4,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap243334e0-84') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG os_vif [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:59:a5:ef,bridge_name='br-int',has_traffic_filtering=True,id=243334e0-8471-4569-bb6a-38c530471bb4,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap243334e0-84') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap243334e0-84, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap243334e0-84, col_values=(('external_ids', {'iface-id': '243334e0-8471-4569-bb6a-38c530471bb4', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:59:a5:ef', 'vm-uuid': '313835fa-6405-41ba-bc89-c4bfc119326a'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:06 user nova-compute[71317]: INFO os_vif [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:59:a5:ef,bridge_name='br-int',has_traffic_filtering=True,id=243334e0-8471-4569-bb6a-38c530471bb4,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap243334e0-84')
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] No VIF found with MAC fa:16:3e:59:a5:ef, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] No BDM found with device name hda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:19:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] No VIF found with MAC fa:16:3e:58:93:64, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:19:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6a8c013-14c2-49e1-8cc1-32a009845707 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.697s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:07 user nova-compute[71317]: DEBUG nova.network.neutron [req-42c0f7f2-143e-4596-b8f7-33d7d46c1d50 req-b2ea00c2-bf7b-495e-8bec-04e1eff57ffd service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Updated VIF entry in instance network info cache for port 243334e0-8471-4569-bb6a-38c530471bb4. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:19:07 user nova-compute[71317]: DEBUG nova.network.neutron [req-42c0f7f2-143e-4596-b8f7-33d7d46c1d50 req-b2ea00c2-bf7b-495e-8bec-04e1eff57ffd service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Updating instance_info_cache with network_info: [{"id": "243334e0-8471-4569-bb6a-38c530471bb4", "address": "fa:16:3e:59:a5:ef", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap243334e0-84", "ovs_interfaceid": "243334e0-8471-4569-bb6a-38c530471bb4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:19:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-42c0f7f2-143e-4596-b8f7-33d7d46c1d50 req-b2ea00c2-bf7b-495e-8bec-04e1eff57ffd service nova] Releasing lock "refresh_cache-313835fa-6405-41ba-bc89-c4bfc119326a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:19:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:07 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lazy-loading 'flavor' on Instance uuid e5344b0d-2d73-40c7-8ac3-535396e985a4 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a576016a-f803-48c8-9e25-e783e410bdf5 tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 3.091s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:08 user nova-compute[71317]: INFO nova.compute.manager [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Terminating instance
Apr 14 04:19:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PuAKITjr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:08 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:08 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:08 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc8a459a-282d-4e2d-bb75-5cd0b9d7920b]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:19:08 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b919857c-a511-4fb2-b86d-2f2ad26ac9d6]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd] added
Apr 14 04:19:08 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1db7dd08-0910-43d9-b176-f1d6c94ed070]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e80f6c75-00e2-4c01-8272-b530c3f9530b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v PuAKITjr {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v PuAKITjr" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f373ec0f-52a4-4b95-8082-ec4b6cba0640]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'PuAKITjr'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c578fa9-1a18-42e7-9097-6694009ddf83]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[83274515-63c4-4c73-bae0-fce430af966c]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-668e773b-45b3-471a-9cc1-68bc391d5a8d req-88313ff8-ac5c-4da9-b6e0-1603ad4c4ba3 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Received event network-vif-unplugged-2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-668e773b-45b3-471a-9cc1-68bc391d5a8d req-88313ff8-ac5c-4da9-b6e0-1603ad4c4ba3 service nova] Acquiring lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-668e773b-45b3-471a-9cc1-68bc391d5a8d req-88313ff8-ac5c-4da9-b6e0-1603ad4c4ba3 service nova] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-668e773b-45b3-471a-9cc1-68bc391d5a8d req-88313ff8-ac5c-4da9-b6e0-1603ad4c4ba3 service nova] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-668e773b-45b3-471a-9cc1-68bc391d5a8d req-88313ff8-ac5c-4da9-b6e0-1603ad4c4ba3 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] No waiting events found dispatching network-vif-unplugged-2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-668e773b-45b3-471a-9cc1-68bc391d5a8d req-88313ff8-ac5c-4da9-b6e0-1603ad4c4ba3 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Received event network-vif-unplugged-2d24208b-94ce-4364-86a4-1758c24e9047 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --login" returned: 0 in 0.094s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[79d7767b-9cd8-4c1f-b9c6-d560c16e713f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260]
Apr 14 04:19:08 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:08 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[59b23f3f-4f86-4311-ba31-218aa07d0df3]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[030c9f50-9427-4cd0-a082-d9531eb37700]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:08 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)
Apr 14 04:19:08 user nova-compute[71317]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:08 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.326s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] HCTL ('37', '-', '-', 0) found on session 9 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[81a9586a-6374-4727-8e15-6e0eaa809005]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Searching for a device in session 9 and hctl ['37', '*', '*', 0] yield: None {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:08 user nova-compute[71317]: INFO nova.compute.manager [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Detaching volume 80a9f7e6-e364-4c2b-9814-96061ad47dd6
Apr 14 04:19:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Instance destroyed successfully.
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.objects.instance [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lazy-loading 'resources' on Instance uuid e5344b0d-2d73-40c7-8ac3-535396e985a4 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-1544950554',display_name='tempest-AttachSCSIVolumeTestJSON-server-1544950554',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-1544950554',id=9,image_ref='f478712d-cb56-468b-a1b1-14aa5eab7104',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP2a/q8z148VuqmzCdTppqcm7Q47cCuI3B0oR8ww+C0FRQKet9rnqUXrh5enWkvTgHjdhAOOgHwD+H7zMvAp4yzILDVjHd5SWnmFQ8IxBan80X7GvaW+XLPf39HLZ/j3xA==',key_name='tempest-keypair-2075478387',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='e0e636ba0ee9486f9f48905d886ab547',ramdisk_id='',reservation_id='r-vkwun6nu',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='f478712d-cb56-468b-a1b1-14aa5eab7104',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_scsi_model='virtio-scsi',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-AttachSCSIVolumeTestJSON-1503158328',owner_user_name='tempest-AttachSCSIVolumeTestJSON-1503158328-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:17:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d1b09bc7a0344ae782d33d3222a8ea88',uuid=e5344b0d-2d73-40c7-8ac3-535396e985a4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2d24208b-94ce-4364-86a4-1758c24e9047", "address": "fa:16:3e:22:c7:b3", "network": {"id": "69963625-cff2-4260-908f-82359d8b2f77", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-558617934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.176", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e0e636ba0ee9486f9f48905d886ab547", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d24208b-94", "ovs_interfaceid": "2d24208b-94ce-4364-86a4-1758c24e9047", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Converting VIF {"id": "2d24208b-94ce-4364-86a4-1758c24e9047", "address": "fa:16:3e:22:c7:b3", "network": {"id": "69963625-cff2-4260-908f-82359d8b2f77", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-558617934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.176", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e0e636ba0ee9486f9f48905d886ab547", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d24208b-94", "ovs_interfaceid": "2d24208b-94ce-4364-86a4-1758c24e9047", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:22:c7:b3,bridge_name='br-int',has_traffic_filtering=True,id=2d24208b-94ce-4364-86a4-1758c24e9047,network=Network(69963625-cff2-4260-908f-82359d8b2f77),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d24208b-94') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_vif [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:22:c7:b3,bridge_name='br-int',has_traffic_filtering=True,id=2d24208b-94ce-4364-86a4-1758c24e9047,network=Network(69963625-cff2-4260-908f-82359d8b2f77),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d24208b-94') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2d24208b-94, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:19:09 user nova-compute[71317]: INFO nova.virt.block_device [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Attempting to driver detach volume 80a9f7e6-e364-4c2b-9814-96061ad47dd6 from mountpoint /dev/vdb
Apr 14 04:19:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:09 user nova-compute[71317]: INFO os_vif [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:22:c7:b3,bridge_name='br-int',has_traffic_filtering=True,id=2d24208b-94ce-4364-86a4-1758c24e9047,network=Network(69963625-cff2-4260-908f-82359d8b2f77),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d24208b-94')
Apr 14 04:19:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Deleting instance files /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4_del
Apr 14 04:19:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Deletion of /opt/stack/data/nova/instances/e5344b0d-2d73-40c7-8ac3-535396e985a4_del complete
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Attempting to detach device vdb from instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:19:09 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:19:09 user nova-compute[71317]:   <source dev="/dev/sdg"/>
Apr 14 04:19:09 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:19:09 user nova-compute[71317]:   <serial>80a9f7e6-e364-4c2b-9814-96061ad47dd6</serial>
Apr 14 04:19:09 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Apr 14 04:19:09 user nova-compute[71317]: </disk>
Apr 14 04:19:09 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:19:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Successfully detached device vdb from instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 from the persistent domain config.
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:19:09 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:19:09 user nova-compute[71317]:   <source dev="/dev/sdg"/>
Apr 14 04:19:09 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:19:09 user nova-compute[71317]:   <serial>80a9f7e6-e364-4c2b-9814-96061ad47dd6</serial>
Apr 14 04:19:09 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Apr 14 04:19:09 user nova-compute[71317]: </disk>
Apr 14 04:19:09 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:19:09 user nova-compute[71317]: INFO nova.compute.manager [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Took 0.90 seconds to destroy the instance on the hypervisor.
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681445949.584557, 8ed1dea8-e48a-49f7-924e-82cf639a1677 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:19:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Successfully detached device vdb from instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 from the live domain config.
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NtrRMlgi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:09 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Searching for a device in session 9 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PuAKITjr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.358s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] <== connect_volume: return (1359ms) {'type': 'block', 'scsi_wwn': '23330303765366434', 'path': '/dev/sdf'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23330303765366434', 'path': '/dev/sdf'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.232s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.objects.instance [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lazy-loading 'flavor' on Instance uuid b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8a404f2-adb1-4289-b9de-676557ad5a6e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:19:09 user nova-compute[71317]: No targets found.
Apr 14 04:19:09 user nova-compute[71317]: iSNS:
Apr 14 04:19:09 user nova-compute[71317]: No targets found.
Apr 14 04:19:09 user nova-compute[71317]: STATIC:
Apr 14 04:19:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282
Apr 14 04:19:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd
Apr 14 04:19:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6
Apr 14 04:19:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b
Apr 14 04:19:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:19:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3
Apr 14 04:19:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:09 user nova-compute[71317]: FIRMWARE:
Apr 14 04:19:09 user nova-compute[71317]: No targets found.
Apr 14 04:19:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:19:09 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:19:09 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:19:09 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:19:09 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:19:09 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:19:09 user nova-compute[71317]:   <serial>f3255f25-0334-46dd-a514-12a56a1deffd</serial>
Apr 14 04:19:09 user nova-compute[71317]: </disk>
Apr 14 04:19:09 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcffba1a-842f-4083-80ee-950f859d6b3a]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4945a988-de76-4b95-adbc-dd55b02dc5eb]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:09 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:09 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:09 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:09 user nova-compute[71317]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)
Apr 14 04:19:09 user nova-compute[71317]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:09 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:09 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:09 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:09 user nova-compute[71317]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 (non-flash)
Apr 14 04:19:09 user nova-compute[71317]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f849076ab00>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6'): ({'sdg'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Removing single pathed devices sdg {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b17db17-302a-4743-b1dc-0228e411142d]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c8cf3ee-15ab-4539-83af-e983496b83e9]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Flushing IO for device /dev/sdg {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2b6ce11-3194-4a85-b766-1d4a6564d00e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:19:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.048s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3fe76de5-616b-4c67-b101-c519dba074b3]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] SCSI volumes sdg have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1bd06c7c-2373-4465-a4e1-96ab4441842d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --logout" returned: 0 in 0.060s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1850ea46-b541-42f4-b32a-604492d4e1a2]: (4, ('Logging out of session [sid: 8, target: iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6, portal: 172.16.0.220,3260]\nLogout of [sid: 8, target: iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 8, target: iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6, portal: 172.16.0.220,3260]
Apr 14 04:19:10 user nova-compute[71317]: Logout of [sid: 8, target: iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80a9f7e6-e364-4c2b-9814-96061ad47dd6 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.037s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a732b3d2-7da2-4fbd-9167-f065819eaaee]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.352s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] <== disconnect_volume: return (584ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-31225ef8-5c3f-49f1-bd52-d39bcbea3f1e req-f7f6f835-b2cb-41e7-be12-0ffdc7287557 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Received event network-vif-deleted-2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:10 user nova-compute[71317]: INFO nova.compute.manager [req-31225ef8-5c3f-49f1-bd52-d39bcbea3f1e req-f7f6f835-b2cb-41e7-be12-0ffdc7287557 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Neutron deleted interface 2d24208b-94ce-4364-86a4-1758c24e9047; detaching it from the instance and deleting it from the info cache
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.network.neutron [req-31225ef8-5c3f-49f1-bd52-d39bcbea3f1e req-f7f6f835-b2cb-41e7-be12-0ffdc7287557 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:19:10 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Took 1.12 seconds to deallocate network for instance.
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-31225ef8-5c3f-49f1-bd52-d39bcbea3f1e req-f7f6f835-b2cb-41e7-be12-0ffdc7287557 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Detach interface failed, port_id=2d24208b-94ce-4364-86a4-1758c24e9047, reason: Instance e5344b0d-2d73-40c7-8ac3-535396e985a4 could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] No VIF found with MAC fa:16:3e:cd:d0:ea, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9369e401-f616-43aa-82ff-882f10e94f56 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.976s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445950.479263, 313835fa-6405-41ba-bc89-c4bfc119326a => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:19:10 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] VM Resumed (Lifecycle Event)
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:19:10 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Instance spawned successfully.
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:19:10 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681445950.4821792, 313835fa-6405-41ba-bc89-c4bfc119326a => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:19:10 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] VM Started (Lifecycle Event)
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:19:10 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:19:10 user nova-compute[71317]: INFO nova.compute.manager [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Took 5.64 seconds to spawn the instance on the hypervisor.
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:19:10 user nova-compute[71317]: INFO nova.compute.manager [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Took 6.45 seconds to build instance.
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-76b81920-c20f-4a9c-acba-50fdc8a92691 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "313835fa-6405-41ba-bc89-c4bfc119326a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.556s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Received event network-vif-plugged-2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] Acquiring lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] No waiting events found dispatching network-vif-plugged-2d24208b-94ce-4364-86a4-1758c24e9047 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:10 user nova-compute[71317]: WARNING nova.compute.manager [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Received unexpected event network-vif-plugged-2d24208b-94ce-4364-86a4-1758c24e9047 for instance with vm_state deleted and task_state None.
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Received event network-vif-plugged-243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] Acquiring lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] Lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] Lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] No waiting events found dispatching network-vif-plugged-243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:10 user nova-compute[71317]: WARNING nova.compute.manager [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Received unexpected event network-vif-plugged-243334e0-8471-4569-bb6a-38c530471bb4 for instance with vm_state active and task_state None.
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Received event network-vif-plugged-243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] Acquiring lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] Lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] Lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] No waiting events found dispatching network-vif-plugged-243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:10 user nova-compute[71317]: WARNING nova.compute.manager [req-36c387b3-a42a-419d-944c-422baa575151 req-6195e96b-cfe3-4927-851a-c00ad9c17a99 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Received unexpected event network-vif-plugged-243334e0-8471-4569-bb6a-38c530471bb4 for instance with vm_state active and task_state None.
Apr 14 04:19:10 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.477s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:10 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Deleted allocations for instance e5344b0d-2d73-40c7-8ac3-535396e985a4
Apr 14 04:19:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-93f8b7e8-23e9-4da8-8ad3-0944afcee77e tempest-AttachSCSIVolumeTestJSON-1503158328 tempest-AttachSCSIVolumeTestJSON-1503158328-project-member] Lock "e5344b0d-2d73-40c7-8ac3-535396e985a4" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.716s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lazy-loading 'flavor' on Instance uuid 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0ff8fb13-c821-49ed-bb1a-b21e6652ab26 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.151s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "interface-8ed1dea8-e48a-49f7-924e-82cf639a1677-25048f42-cfe5-48d1-8620-ac6ac6aa9b82" by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "interface-8ed1dea8-e48a-49f7-924e-82cf639a1677-25048f42-cfe5-48d1-8620-ac6ac6aa9b82" acquired by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lazy-loading 'flavor' on Instance uuid 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=InstanceDeviceMetadata,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-420804238',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-420804238',id=10,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBvOf9hss+Yqylm6PnX6la1ymm/55r9GRsHJ0NsNe4bgFVG/3wZy3kBc49f/xLbrJ51dzLLHr/o6IQVXcYhlDKHgp4DCKWnVv/anFtE+/Na/vdZgAYvQkLcZYHFyD4dFRQ==',key_name='tempest-keypair-330344245',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='01cefa66350a4432ad71204b39e953be',ramdisk_id='',reservation_id='r-p91nk3xp',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-926384577',owner_user_name='tempest-TaggedAttachmentsTest-926384577-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:17:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='0aeeeae7cbc340daa8cd81c1418794e0',uuid=8ed1dea8-e48a-49f7-924e-82cf639a1677,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converting VIF {"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:14:36:0b,bridge_name='br-int',has_traffic_filtering=True,id=25048f42-cfe5-48d1-8620-ac6ac6aa9b82,network=Network(d28017dc-3976-4caa-a881-077e38a8b0c7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap25048f42-cf') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:14:36:0b"/><model type="virtio"/><mtu size="1442"/><target dev="tap25048f42-cf"/></interface> {{(pid=71317) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:14:36:0b"/><model type="virtio"/><mtu size="1442"/><target dev="tap25048f42-cf"/></interface> {{(pid=71317) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Attempting to detach device tap25048f42-cf from instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] detach device xml: <interface type="ethernet">
Apr 14 04:19:11 user nova-compute[71317]:   <mac address="fa:16:3e:14:36:0b"/>
Apr 14 04:19:11 user nova-compute[71317]:   <model type="virtio"/>
Apr 14 04:19:11 user nova-compute[71317]:   <mtu size="1442"/>
Apr 14 04:19:11 user nova-compute[71317]:   <target dev="tap25048f42-cf"/>
Apr 14 04:19:11 user nova-compute[71317]: </interface>
Apr 14 04:19:11 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:14:36:0b"/><model type="virtio"/><mtu size="1442"/><target dev="tap25048f42-cf"/></interface> {{(pid=71317) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] interface for config: <interface type="ethernet"><mac address="fa:16:3e:14:36:0b"/><model type="virtio"/><mtu size="1442"/><target dev="tap25048f42-cf"/></interface>not found in domain: <domain type='kvm' id='10'>
Apr 14 04:19:11 user nova-compute[71317]:   <name>instance-0000000a</name>
Apr 14 04:19:11 user nova-compute[71317]:   <uuid>8ed1dea8-e48a-49f7-924e-82cf639a1677</uuid>
Apr 14 04:19:11 user nova-compute[71317]:   <metadata>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:19:11 user nova-compute[71317]:   <nova:package version="0.0.0"/>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:name>tempest-device-tagging-server-420804238</nova:name>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:creationTime>2023-04-14 04:18:59</nova:creationTime>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:flavor name="m1.nano">
Apr 14 04:19:11 user nova-compute[71317]:     <nova:memory>128</nova:memory>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:disk>1</nova:disk>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:swap>0</nova:swap>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:vcpus>1</nova:vcpus>
Apr 14 04:19:11 user nova-compute[71317]:   </nova:flavor>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:owner>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:user uuid="0aeeeae7cbc340daa8cd81c1418794e0">tempest-TaggedAttachmentsTest-926384577-project-member</nova:user>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:project uuid="01cefa66350a4432ad71204b39e953be">tempest-TaggedAttachmentsTest-926384577</nova:project>
Apr 14 04:19:11 user nova-compute[71317]:   </nova:owner>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:ports>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:port uuid="60579bc4-de13-48d7-b42d-2b9fc5597e45">
Apr 14 04:19:11 user nova-compute[71317]:       <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 04:19:11 user nova-compute[71317]:     </nova:port>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:port uuid="25048f42-cfe5-48d1-8620-ac6ac6aa9b82">
Apr 14 04:19:11 user nova-compute[71317]:       <nova:ip type="fixed" address="10.10.10.130" ipVersion="4"/>
Apr 14 04:19:11 user nova-compute[71317]:     </nova:port>
Apr 14 04:19:11 user nova-compute[71317]:   </nova:ports>
Apr 14 04:19:11 user nova-compute[71317]: </nova:instance>
Apr 14 04:19:11 user nova-compute[71317]:   </metadata>
Apr 14 04:19:11 user nova-compute[71317]:   <memory unit='KiB'>131072</memory>
Apr 14 04:19:11 user nova-compute[71317]:   <currentMemory unit='KiB'>131072</currentMemory>
Apr 14 04:19:11 user nova-compute[71317]:   <vcpu placement='static'>1</vcpu>
Apr 14 04:19:11 user nova-compute[71317]:   <resource>
Apr 14 04:19:11 user nova-compute[71317]:     <partition>/machine</partition>
Apr 14 04:19:11 user nova-compute[71317]:   </resource>
Apr 14 04:19:11 user nova-compute[71317]:   <sysinfo type='smbios'>
Apr 14 04:19:11 user nova-compute[71317]:     <system>
Apr 14 04:19:11 user nova-compute[71317]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Apr 14 04:19:11 user nova-compute[71317]:       <entry name='product'>OpenStack Nova</entry>
Apr 14 04:19:11 user nova-compute[71317]:       <entry name='version'>0.0.0</entry>
Apr 14 04:19:11 user nova-compute[71317]:       <entry name='serial'>8ed1dea8-e48a-49f7-924e-82cf639a1677</entry>
Apr 14 04:19:11 user nova-compute[71317]:       <entry name='uuid'>8ed1dea8-e48a-49f7-924e-82cf639a1677</entry>
Apr 14 04:19:11 user nova-compute[71317]:       <entry name='family'>Virtual Machine</entry>
Apr 14 04:19:11 user nova-compute[71317]:     </system>
Apr 14 04:19:11 user nova-compute[71317]:   </sysinfo>
Apr 14 04:19:11 user nova-compute[71317]:   <os>
Apr 14 04:19:11 user nova-compute[71317]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Apr 14 04:19:11 user nova-compute[71317]:     <boot dev='hd'/>
Apr 14 04:19:11 user nova-compute[71317]:     <smbios mode='sysinfo'/>
Apr 14 04:19:11 user nova-compute[71317]:   </os>
Apr 14 04:19:11 user nova-compute[71317]:   <features>
Apr 14 04:19:11 user nova-compute[71317]:     <acpi/>
Apr 14 04:19:11 user nova-compute[71317]:     <apic/>
Apr 14 04:19:11 user nova-compute[71317]:     <vmcoreinfo state='on'/>
Apr 14 04:19:11 user nova-compute[71317]:   </features>
Apr 14 04:19:11 user nova-compute[71317]:   <cpu mode='custom' match='exact' check='full'>
Apr 14 04:19:11 user nova-compute[71317]:     <model fallback='forbid'>Nehalem</model>
Apr 14 04:19:11 user nova-compute[71317]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Apr 14 04:19:11 user nova-compute[71317]:     <feature policy='require' name='vme'/>
Apr 14 04:19:11 user nova-compute[71317]:     <feature policy='require' name='x2apic'/>
Apr 14 04:19:11 user nova-compute[71317]:     <feature policy='require' name='hypervisor'/>
Apr 14 04:19:11 user nova-compute[71317]:   </cpu>
Apr 14 04:19:11 user nova-compute[71317]:   <clock offset='utc'>
Apr 14 04:19:11 user nova-compute[71317]:     <timer name='pit' tickpolicy='delay'/>
Apr 14 04:19:11 user nova-compute[71317]:     <timer name='rtc' tickpolicy='catchup'/>
Apr 14 04:19:11 user nova-compute[71317]:     <timer name='hpet' present='no'/>
Apr 14 04:19:11 user nova-compute[71317]:   </clock>
Apr 14 04:19:11 user nova-compute[71317]:   <on_poweroff>destroy</on_poweroff>
Apr 14 04:19:11 user nova-compute[71317]:   <on_reboot>restart</on_reboot>
Apr 14 04:19:11 user nova-compute[71317]:   <on_crash>destroy</on_crash>
Apr 14 04:19:11 user nova-compute[71317]:   <devices>
Apr 14 04:19:11 user nova-compute[71317]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 14 04:19:11 user nova-compute[71317]:     <disk type='file' device='disk'>
Apr 14 04:19:11 user nova-compute[71317]:       <driver name='qemu' type='qcow2' cache='none'/>
Apr 14 04:19:11 user nova-compute[71317]:       <source file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk' index='2'/>
Apr 14 04:19:11 user nova-compute[71317]:       <backingStore type='file' index='3'>
Apr 14 04:19:11 user nova-compute[71317]:         <format type='raw'/>
Apr 14 04:19:11 user nova-compute[71317]:         <source file='/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d'/>
Apr 14 04:19:11 user nova-compute[71317]:         <backingStore/>
Apr 14 04:19:11 user nova-compute[71317]:       </backingStore>
Apr 14 04:19:11 user nova-compute[71317]:       <target dev='vda' bus='virtio'/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='virtio-disk0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </disk>
Apr 14 04:19:11 user nova-compute[71317]:     <disk type='file' device='cdrom'>
Apr 14 04:19:11 user nova-compute[71317]:       <driver name='qemu' type='raw' cache='none'/>
Apr 14 04:19:11 user nova-compute[71317]:       <source file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk.config' index='1'/>
Apr 14 04:19:11 user nova-compute[71317]:       <backingStore/>
Apr 14 04:19:11 user nova-compute[71317]:       <target dev='hda' bus='ide'/>
Apr 14 04:19:11 user nova-compute[71317]:       <readonly/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='ide0-0-0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </disk>
Apr 14 04:19:11 user nova-compute[71317]:     <controller type='usb' index='0' model='none'>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='usb'/>
Apr 14 04:19:11 user nova-compute[71317]:     </controller>
Apr 14 04:19:11 user nova-compute[71317]:     <controller type='pci' index='0' model='pci-root'>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='pci.0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </controller>
Apr 14 04:19:11 user nova-compute[71317]:     <controller type='ide' index='0'>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='ide'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Apr 14 04:19:11 user nova-compute[71317]:     </controller>
Apr 14 04:19:11 user nova-compute[71317]:     <interface type='ethernet'>
Apr 14 04:19:11 user nova-compute[71317]:       <mac address='fa:16:3e:58:93:64'/>
Apr 14 04:19:11 user nova-compute[71317]:       <target dev='tap60579bc4-de'/>
Apr 14 04:19:11 user nova-compute[71317]:       <model type='virtio'/>
Apr 14 04:19:11 user nova-compute[71317]:       <mtu size='1442'/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='net0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </interface>
Apr 14 04:19:11 user nova-compute[71317]:     <interface type='ethernet'>
Apr 14 04:19:11 user nova-compute[71317]:       <mac address='fa:16:3e:14:36:0b'/>
Apr 14 04:19:11 user nova-compute[71317]:       <target dev='tap25048f42-cf'/>
Apr 14 04:19:11 user nova-compute[71317]:       <model type='virtio'/>
Apr 14 04:19:11 user nova-compute[71317]:       <mtu size='1442'/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='net1'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </interface>
Apr 14 04:19:11 user nova-compute[71317]:     <serial type='pty'>
Apr 14 04:19:11 user nova-compute[71317]:       <source path='/dev/pts/9'/>
Apr 14 04:19:11 user nova-compute[71317]:       <log file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/console.log' append='off'/>
Apr 14 04:19:11 user nova-compute[71317]:       <target type='isa-serial' port='0'>
Apr 14 04:19:11 user nova-compute[71317]:         <model name='isa-serial'/>
Apr 14 04:19:11 user nova-compute[71317]:       </target>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='serial0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </serial>
Apr 14 04:19:11 user nova-compute[71317]:     <console type='pty' tty='/dev/pts/9'>
Apr 14 04:19:11 user nova-compute[71317]:       <source path='/dev/pts/9'/>
Apr 14 04:19:11 user nova-compute[71317]:       <log file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/console.log' append='off'/>
Apr 14 04:19:11 user nova-compute[71317]:       <target type='serial' port='0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='serial0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </console>
Apr 14 04:19:11 user nova-compute[71317]:     <input type='mouse' bus='ps2'>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='input0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </input>
Apr 14 04:19:11 user nova-compute[71317]:     <input type='keyboard' bus='ps2'>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='input1'/>
Apr 14 04:19:11 user nova-compute[71317]:     </input>
Apr 14 04:19:11 user nova-compute[71317]:     <graphics type='vnc' port='5909' autoport='yes' listen='0.0.0.0'>
Apr 14 04:19:11 user nova-compute[71317]:       <listen type='address' address='0.0.0.0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </graphics>
Apr 14 04:19:11 user nova-compute[71317]:     <audio id='1' type='none'/>
Apr 14 04:19:11 user nova-compute[71317]:     <video>
Apr 14 04:19:11 user nova-compute[71317]:       <model type='virtio' heads='1' primary='yes'/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='video0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </video>
Apr 14 04:19:11 user nova-compute[71317]:     <memballoon model='virtio'>
Apr 14 04:19:11 user nova-compute[71317]:       <stats period='10'/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='balloon0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </memballoon>
Apr 14 04:19:11 user nova-compute[71317]:     <rng model='virtio'>
Apr 14 04:19:11 user nova-compute[71317]:       <backend model='random'>/dev/urandom</backend>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='rng0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </rng>
Apr 14 04:19:11 user nova-compute[71317]:   </devices>
Apr 14 04:19:11 user nova-compute[71317]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Apr 14 04:19:11 user nova-compute[71317]:     <label>libvirt-8ed1dea8-e48a-49f7-924e-82cf639a1677</label>
Apr 14 04:19:11 user nova-compute[71317]:     <imagelabel>libvirt-8ed1dea8-e48a-49f7-924e-82cf639a1677</imagelabel>
Apr 14 04:19:11 user nova-compute[71317]:   </seclabel>
Apr 14 04:19:11 user nova-compute[71317]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Apr 14 04:19:11 user nova-compute[71317]:     <label>+64055:+108</label>
Apr 14 04:19:11 user nova-compute[71317]:     <imagelabel>+64055:+108</imagelabel>
Apr 14 04:19:11 user nova-compute[71317]:   </seclabel>
Apr 14 04:19:11 user nova-compute[71317]: </domain>
Apr 14 04:19:11 user nova-compute[71317]:  {{(pid=71317) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Apr 14 04:19:11 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Successfully detached device tap25048f42-cf from instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 from the persistent domain config.
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] (1/8): Attempting to detach device tap25048f42-cf with device alias net1 from instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] detach device xml: <interface type="ethernet">
Apr 14 04:19:11 user nova-compute[71317]:   <mac address="fa:16:3e:14:36:0b"/>
Apr 14 04:19:11 user nova-compute[71317]:   <model type="virtio"/>
Apr 14 04:19:11 user nova-compute[71317]:   <mtu size="1442"/>
Apr 14 04:19:11 user nova-compute[71317]:   <target dev="tap25048f42-cf"/>
Apr 14 04:19:11 user nova-compute[71317]: </interface>
Apr 14 04:19:11 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681445951.5926237, 8ed1dea8-e48a-49f7-924e-82cf639a1677 => net1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Start waiting for the detach event from libvirt for device tap25048f42-cf with device alias net1 for instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:14:36:0b"/><model type="virtio"/><mtu size="1442"/><target dev="tap25048f42-cf"/></interface> {{(pid=71317) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] interface for config: <interface type="ethernet"><mac address="fa:16:3e:14:36:0b"/><model type="virtio"/><mtu size="1442"/><target dev="tap25048f42-cf"/></interface>not found in domain: <domain type='kvm' id='10'>
Apr 14 04:19:11 user nova-compute[71317]:   <name>instance-0000000a</name>
Apr 14 04:19:11 user nova-compute[71317]:   <uuid>8ed1dea8-e48a-49f7-924e-82cf639a1677</uuid>
Apr 14 04:19:11 user nova-compute[71317]:   <metadata>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:19:11 user nova-compute[71317]:   <nova:package version="0.0.0"/>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:name>tempest-device-tagging-server-420804238</nova:name>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:creationTime>2023-04-14 04:18:59</nova:creationTime>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:flavor name="m1.nano">
Apr 14 04:19:11 user nova-compute[71317]:     <nova:memory>128</nova:memory>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:disk>1</nova:disk>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:swap>0</nova:swap>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:vcpus>1</nova:vcpus>
Apr 14 04:19:11 user nova-compute[71317]:   </nova:flavor>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:owner>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:user uuid="0aeeeae7cbc340daa8cd81c1418794e0">tempest-TaggedAttachmentsTest-926384577-project-member</nova:user>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:project uuid="01cefa66350a4432ad71204b39e953be">tempest-TaggedAttachmentsTest-926384577</nova:project>
Apr 14 04:19:11 user nova-compute[71317]:   </nova:owner>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:ports>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:port uuid="60579bc4-de13-48d7-b42d-2b9fc5597e45">
Apr 14 04:19:11 user nova-compute[71317]:       <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 04:19:11 user nova-compute[71317]:     </nova:port>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:port uuid="25048f42-cfe5-48d1-8620-ac6ac6aa9b82">
Apr 14 04:19:11 user nova-compute[71317]:       <nova:ip type="fixed" address="10.10.10.130" ipVersion="4"/>
Apr 14 04:19:11 user nova-compute[71317]:     </nova:port>
Apr 14 04:19:11 user nova-compute[71317]:   </nova:ports>
Apr 14 04:19:11 user nova-compute[71317]: </nova:instance>
Apr 14 04:19:11 user nova-compute[71317]:   </metadata>
Apr 14 04:19:11 user nova-compute[71317]:   <memory unit='KiB'>131072</memory>
Apr 14 04:19:11 user nova-compute[71317]:   <currentMemory unit='KiB'>131072</currentMemory>
Apr 14 04:19:11 user nova-compute[71317]:   <vcpu placement='static'>1</vcpu>
Apr 14 04:19:11 user nova-compute[71317]:   <resource>
Apr 14 04:19:11 user nova-compute[71317]:     <partition>/machine</partition>
Apr 14 04:19:11 user nova-compute[71317]:   </resource>
Apr 14 04:19:11 user nova-compute[71317]:   <sysinfo type='smbios'>
Apr 14 04:19:11 user nova-compute[71317]:     <system>
Apr 14 04:19:11 user nova-compute[71317]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Apr 14 04:19:11 user nova-compute[71317]:       <entry name='product'>OpenStack Nova</entry>
Apr 14 04:19:11 user nova-compute[71317]:       <entry name='version'>0.0.0</entry>
Apr 14 04:19:11 user nova-compute[71317]:       <entry name='serial'>8ed1dea8-e48a-49f7-924e-82cf639a1677</entry>
Apr 14 04:19:11 user nova-compute[71317]:       <entry name='uuid'>8ed1dea8-e48a-49f7-924e-82cf639a1677</entry>
Apr 14 04:19:11 user nova-compute[71317]:       <entry name='family'>Virtual Machine</entry>
Apr 14 04:19:11 user nova-compute[71317]:     </system>
Apr 14 04:19:11 user nova-compute[71317]:   </sysinfo>
Apr 14 04:19:11 user nova-compute[71317]:   <os>
Apr 14 04:19:11 user nova-compute[71317]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Apr 14 04:19:11 user nova-compute[71317]:     <boot dev='hd'/>
Apr 14 04:19:11 user nova-compute[71317]:     <smbios mode='sysinfo'/>
Apr 14 04:19:11 user nova-compute[71317]:   </os>
Apr 14 04:19:11 user nova-compute[71317]:   <features>
Apr 14 04:19:11 user nova-compute[71317]:     <acpi/>
Apr 14 04:19:11 user nova-compute[71317]:     <apic/>
Apr 14 04:19:11 user nova-compute[71317]:     <vmcoreinfo state='on'/>
Apr 14 04:19:11 user nova-compute[71317]:   </features>
Apr 14 04:19:11 user nova-compute[71317]:   <cpu mode='custom' match='exact' check='full'>
Apr 14 04:19:11 user nova-compute[71317]:     <model fallback='forbid'>Nehalem</model>
Apr 14 04:19:11 user nova-compute[71317]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Apr 14 04:19:11 user nova-compute[71317]:     <feature policy='require' name='vme'/>
Apr 14 04:19:11 user nova-compute[71317]:     <feature policy='require' name='x2apic'/>
Apr 14 04:19:11 user nova-compute[71317]:     <feature policy='require' name='hypervisor'/>
Apr 14 04:19:11 user nova-compute[71317]:   </cpu>
Apr 14 04:19:11 user nova-compute[71317]:   <clock offset='utc'>
Apr 14 04:19:11 user nova-compute[71317]:     <timer name='pit' tickpolicy='delay'/>
Apr 14 04:19:11 user nova-compute[71317]:     <timer name='rtc' tickpolicy='catchup'/>
Apr 14 04:19:11 user nova-compute[71317]:     <timer name='hpet' present='no'/>
Apr 14 04:19:11 user nova-compute[71317]:   </clock>
Apr 14 04:19:11 user nova-compute[71317]:   <on_poweroff>destroy</on_poweroff>
Apr 14 04:19:11 user nova-compute[71317]:   <on_reboot>restart</on_reboot>
Apr 14 04:19:11 user nova-compute[71317]:   <on_crash>destroy</on_crash>
Apr 14 04:19:11 user nova-compute[71317]:   <devices>
Apr 14 04:19:11 user nova-compute[71317]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 14 04:19:11 user nova-compute[71317]:     <disk type='file' device='disk'>
Apr 14 04:19:11 user nova-compute[71317]:       <driver name='qemu' type='qcow2' cache='none'/>
Apr 14 04:19:11 user nova-compute[71317]:       <source file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk' index='2'/>
Apr 14 04:19:11 user nova-compute[71317]:       <backingStore type='file' index='3'>
Apr 14 04:19:11 user nova-compute[71317]:         <format type='raw'/>
Apr 14 04:19:11 user nova-compute[71317]:         <source file='/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d'/>
Apr 14 04:19:11 user nova-compute[71317]:         <backingStore/>
Apr 14 04:19:11 user nova-compute[71317]:       </backingStore>
Apr 14 04:19:11 user nova-compute[71317]:       <target dev='vda' bus='virtio'/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='virtio-disk0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </disk>
Apr 14 04:19:11 user nova-compute[71317]:     <disk type='file' device='cdrom'>
Apr 14 04:19:11 user nova-compute[71317]:       <driver name='qemu' type='raw' cache='none'/>
Apr 14 04:19:11 user nova-compute[71317]:       <source file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk.config' index='1'/>
Apr 14 04:19:11 user nova-compute[71317]:       <backingStore/>
Apr 14 04:19:11 user nova-compute[71317]:       <target dev='hda' bus='ide'/>
Apr 14 04:19:11 user nova-compute[71317]:       <readonly/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='ide0-0-0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </disk>
Apr 14 04:19:11 user nova-compute[71317]:     <controller type='usb' index='0' model='none'>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='usb'/>
Apr 14 04:19:11 user nova-compute[71317]:     </controller>
Apr 14 04:19:11 user nova-compute[71317]:     <controller type='pci' index='0' model='pci-root'>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='pci.0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </controller>
Apr 14 04:19:11 user nova-compute[71317]:     <controller type='ide' index='0'>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='ide'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Apr 14 04:19:11 user nova-compute[71317]:     </controller>
Apr 14 04:19:11 user nova-compute[71317]:     <interface type='ethernet'>
Apr 14 04:19:11 user nova-compute[71317]:       <mac address='fa:16:3e:58:93:64'/>
Apr 14 04:19:11 user nova-compute[71317]:       <target dev='tap60579bc4-de'/>
Apr 14 04:19:11 user nova-compute[71317]:       <model type='virtio'/>
Apr 14 04:19:11 user nova-compute[71317]:       <mtu size='1442'/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='net0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </interface>
Apr 14 04:19:11 user nova-compute[71317]:     <serial type='pty'>
Apr 14 04:19:11 user nova-compute[71317]:       <source path='/dev/pts/9'/>
Apr 14 04:19:11 user nova-compute[71317]:       <log file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/console.log' append='off'/>
Apr 14 04:19:11 user nova-compute[71317]:       <target type='isa-serial' port='0'>
Apr 14 04:19:11 user nova-compute[71317]:         <model name='isa-serial'/>
Apr 14 04:19:11 user nova-compute[71317]:       </target>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='serial0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </serial>
Apr 14 04:19:11 user nova-compute[71317]:     <console type='pty' tty='/dev/pts/9'>
Apr 14 04:19:11 user nova-compute[71317]:       <source path='/dev/pts/9'/>
Apr 14 04:19:11 user nova-compute[71317]:       <log file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/console.log' append='off'/>
Apr 14 04:19:11 user nova-compute[71317]:       <target type='serial' port='0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='serial0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </console>
Apr 14 04:19:11 user nova-compute[71317]:     <input type='mouse' bus='ps2'>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='input0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </input>
Apr 14 04:19:11 user nova-compute[71317]:     <input type='keyboard' bus='ps2'>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='input1'/>
Apr 14 04:19:11 user nova-compute[71317]:     </input>
Apr 14 04:19:11 user nova-compute[71317]:     <graphics type='vnc' port='5909' autoport='yes' listen='0.0.0.0'>
Apr 14 04:19:11 user nova-compute[71317]:       <listen type='address' address='0.0.0.0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </graphics>
Apr 14 04:19:11 user nova-compute[71317]:     <audio id='1' type='none'/>
Apr 14 04:19:11 user nova-compute[71317]:     <video>
Apr 14 04:19:11 user nova-compute[71317]:       <model type='virtio' heads='1' primary='yes'/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='video0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </video>
Apr 14 04:19:11 user nova-compute[71317]:     <memballoon model='virtio'>
Apr 14 04:19:11 user nova-compute[71317]:       <stats period='10'/>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='balloon0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </memballoon>
Apr 14 04:19:11 user nova-compute[71317]:     <rng model='virtio'>
Apr 14 04:19:11 user nova-compute[71317]:       <backend model='random'>/dev/urandom</backend>
Apr 14 04:19:11 user nova-compute[71317]:       <alias name='rng0'/>
Apr 14 04:19:11 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Apr 14 04:19:11 user nova-compute[71317]:     </rng>
Apr 14 04:19:11 user nova-compute[71317]:   </devices>
Apr 14 04:19:11 user nova-compute[71317]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Apr 14 04:19:11 user nova-compute[71317]:     <label>libvirt-8ed1dea8-e48a-49f7-924e-82cf639a1677</label>
Apr 14 04:19:11 user nova-compute[71317]:     <imagelabel>libvirt-8ed1dea8-e48a-49f7-924e-82cf639a1677</imagelabel>
Apr 14 04:19:11 user nova-compute[71317]:   </seclabel>
Apr 14 04:19:11 user nova-compute[71317]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Apr 14 04:19:11 user nova-compute[71317]:     <label>+64055:+108</label>
Apr 14 04:19:11 user nova-compute[71317]:     <imagelabel>+64055:+108</imagelabel>
Apr 14 04:19:11 user nova-compute[71317]:   </seclabel>
Apr 14 04:19:11 user nova-compute[71317]: </domain>
Apr 14 04:19:11 user nova-compute[71317]:  {{(pid=71317) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Apr 14 04:19:11 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Successfully detached device tap25048f42-cf from instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 from the live domain config.
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=InstanceDeviceMetadata,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-420804238',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-420804238',id=10,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBvOf9hss+Yqylm6PnX6la1ymm/55r9GRsHJ0NsNe4bgFVG/3wZy3kBc49f/xLbrJ51dzLLHr/o6IQVXcYhlDKHgp4DCKWnVv/anFtE+/Na/vdZgAYvQkLcZYHFyD4dFRQ==',key_name='tempest-keypair-330344245',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='01cefa66350a4432ad71204b39e953be',ramdisk_id='',reservation_id='r-p91nk3xp',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-926384577',owner_user_name='tempest-TaggedAttachmentsTest-926384577-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:17:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='0aeeeae7cbc340daa8cd81c1418794e0',uuid=8ed1dea8-e48a-49f7-924e-82cf639a1677,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converting VIF {"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:14:36:0b,bridge_name='br-int',has_traffic_filtering=True,id=25048f42-cfe5-48d1-8620-ac6ac6aa9b82,network=Network(d28017dc-3976-4caa-a881-077e38a8b0c7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap25048f42-cf') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG os_vif [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:14:36:0b,bridge_name='br-int',has_traffic_filtering=True,id=25048f42-cfe5-48d1-8620-ac6ac6aa9b82,network=Network(d28017dc-3976-4caa-a881-077e38a8b0c7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap25048f42-cf') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap25048f42-cf, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:11 user nova-compute[71317]: INFO os_vif [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:14:36:0b,bridge_name='br-int',has_traffic_filtering=True,id=25048f42-cfe5-48d1-8620-ac6ac6aa9b82,network=Network(d28017dc-3976-4caa-a881-077e38a8b0c7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap25048f42-cf')
Apr 14 04:19:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:19:11 user nova-compute[71317]:   <nova:package version="0.0.0"/>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:name>tempest-device-tagging-server-420804238</nova:name>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:creationTime>2023-04-14 04:19:11</nova:creationTime>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:flavor name="m1.nano">
Apr 14 04:19:11 user nova-compute[71317]:     <nova:memory>128</nova:memory>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:disk>1</nova:disk>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:swap>0</nova:swap>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:vcpus>1</nova:vcpus>
Apr 14 04:19:11 user nova-compute[71317]:   </nova:flavor>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:owner>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:user uuid="0aeeeae7cbc340daa8cd81c1418794e0">tempest-TaggedAttachmentsTest-926384577-project-member</nova:user>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:project uuid="01cefa66350a4432ad71204b39e953be">tempest-TaggedAttachmentsTest-926384577</nova:project>
Apr 14 04:19:11 user nova-compute[71317]:   </nova:owner>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:19:11 user nova-compute[71317]:   <nova:ports>
Apr 14 04:19:11 user nova-compute[71317]:     <nova:port uuid="60579bc4-de13-48d7-b42d-2b9fc5597e45">
Apr 14 04:19:11 user nova-compute[71317]:       <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 04:19:11 user nova-compute[71317]:     </nova:port>
Apr 14 04:19:11 user nova-compute[71317]:   </nova:ports>
Apr 14 04:19:11 user nova-compute[71317]: </nova:instance>
Apr 14 04:19:11 user nova-compute[71317]:  {{(pid=71317) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Acquiring lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" acquired by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG os_brick.utils [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5752d84b-3aab-48c2-88f5-b39eb2891198]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[741f7716-daea-473b-a12c-dd5e636139dd]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecbace8d-fa77-44a9-9d13-78ec7dc15849]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ed36380-d7a2-4990-9190-39119d07ded2]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:19:11 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:19:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e93c3105-17d0-4072-bda5-eb00e28d770d]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:19:11 user nova-compute[71317]: DEBUG os_brick.utils [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] <== get_connector_properties: return (139ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:11 user nova-compute[71317]: INFO nova.compute.manager [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Swapping volume f3255f25-0334-46dd-a514-12a56a1deffd for eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370
Apr 14 04:19:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquired lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.network.neutron [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-vif-deleted-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:12 user nova-compute[71317]: INFO nova.compute.manager [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Neutron deleted interface 25048f42-cfe5-48d1-8620-ac6ac6aa9b82; detaching it from the instance and deleting it from the info cache
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.network.neutron [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Updating instance_info_cache with network_info: [{"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.88", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.objects.instance [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] Lazy-loading 'system_metadata' on Instance uuid 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.objects.instance [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] Lazy-loading 'flavor' on Instance uuid 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-420804238',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-420804238',id=10,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBvOf9hss+Yqylm6PnX6la1ymm/55r9GRsHJ0NsNe4bgFVG/3wZy3kBc49f/xLbrJ51dzLLHr/o6IQVXcYhlDKHgp4DCKWnVv/anFtE+/Na/vdZgAYvQkLcZYHFyD4dFRQ==',key_name='tempest-keypair-330344245',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata=<?>,migration_context=None,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='01cefa66350a4432ad71204b39e953be',ramdisk_id='',reservation_id='r-p91nk3xp',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-926384577',owner_user_name='tempest-TaggedAttachmentsTest-926384577-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:17:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='0aeeeae7cbc340daa8cd81c1418794e0',uuid=8ed1dea8-e48a-49f7-924e-82cf639a1677,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] Converting VIF {"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:14:36:0b,bridge_name='br-int',has_traffic_filtering=True,id=25048f42-cfe5-48d1-8620-ac6ac6aa9b82,network=Network(d28017dc-3976-4caa-a881-077e38a8b0c7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap25048f42-cf') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:14:36:0b"/><model type="virtio"/><mtu size="1442"/><target dev="tap25048f42-cf"/></interface> {{(pid=71317) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] interface for config: <interface type="ethernet"><mac address="fa:16:3e:14:36:0b"/><model type="virtio"/><mtu size="1442"/><target dev="tap25048f42-cf"/></interface>not found in domain: <domain type='kvm' id='10'>
Apr 14 04:19:12 user nova-compute[71317]:   <name>instance-0000000a</name>
Apr 14 04:19:12 user nova-compute[71317]:   <uuid>8ed1dea8-e48a-49f7-924e-82cf639a1677</uuid>
Apr 14 04:19:12 user nova-compute[71317]:   <metadata>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:19:12 user nova-compute[71317]:   <nova:package version="0.0.0"/>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:name>tempest-device-tagging-server-420804238</nova:name>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:creationTime>2023-04-14 04:19:11</nova:creationTime>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:flavor name="m1.nano">
Apr 14 04:19:12 user nova-compute[71317]:     <nova:memory>128</nova:memory>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:disk>1</nova:disk>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:swap>0</nova:swap>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:vcpus>1</nova:vcpus>
Apr 14 04:19:12 user nova-compute[71317]:   </nova:flavor>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:owner>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:user uuid="0aeeeae7cbc340daa8cd81c1418794e0">tempest-TaggedAttachmentsTest-926384577-project-member</nova:user>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:project uuid="01cefa66350a4432ad71204b39e953be">tempest-TaggedAttachmentsTest-926384577</nova:project>
Apr 14 04:19:12 user nova-compute[71317]:   </nova:owner>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:ports>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:port uuid="60579bc4-de13-48d7-b42d-2b9fc5597e45">
Apr 14 04:19:12 user nova-compute[71317]:       <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 04:19:12 user nova-compute[71317]:     </nova:port>
Apr 14 04:19:12 user nova-compute[71317]:   </nova:ports>
Apr 14 04:19:12 user nova-compute[71317]: </nova:instance>
Apr 14 04:19:12 user nova-compute[71317]:   </metadata>
Apr 14 04:19:12 user nova-compute[71317]:   <memory unit='KiB'>131072</memory>
Apr 14 04:19:12 user nova-compute[71317]:   <currentMemory unit='KiB'>131072</currentMemory>
Apr 14 04:19:12 user nova-compute[71317]:   <vcpu placement='static'>1</vcpu>
Apr 14 04:19:12 user nova-compute[71317]:   <resource>
Apr 14 04:19:12 user nova-compute[71317]:     <partition>/machine</partition>
Apr 14 04:19:12 user nova-compute[71317]:   </resource>
Apr 14 04:19:12 user nova-compute[71317]:   <sysinfo type='smbios'>
Apr 14 04:19:12 user nova-compute[71317]:     <system>
Apr 14 04:19:12 user nova-compute[71317]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Apr 14 04:19:12 user nova-compute[71317]:       <entry name='product'>OpenStack Nova</entry>
Apr 14 04:19:12 user nova-compute[71317]:       <entry name='version'>0.0.0</entry>
Apr 14 04:19:12 user nova-compute[71317]:       <entry name='serial'>8ed1dea8-e48a-49f7-924e-82cf639a1677</entry>
Apr 14 04:19:12 user nova-compute[71317]:       <entry name='uuid'>8ed1dea8-e48a-49f7-924e-82cf639a1677</entry>
Apr 14 04:19:12 user nova-compute[71317]:       <entry name='family'>Virtual Machine</entry>
Apr 14 04:19:12 user nova-compute[71317]:     </system>
Apr 14 04:19:12 user nova-compute[71317]:   </sysinfo>
Apr 14 04:19:12 user nova-compute[71317]:   <os>
Apr 14 04:19:12 user nova-compute[71317]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Apr 14 04:19:12 user nova-compute[71317]:     <boot dev='hd'/>
Apr 14 04:19:12 user nova-compute[71317]:     <smbios mode='sysinfo'/>
Apr 14 04:19:12 user nova-compute[71317]:   </os>
Apr 14 04:19:12 user nova-compute[71317]:   <features>
Apr 14 04:19:12 user nova-compute[71317]:     <acpi/>
Apr 14 04:19:12 user nova-compute[71317]:     <apic/>
Apr 14 04:19:12 user nova-compute[71317]:     <vmcoreinfo state='on'/>
Apr 14 04:19:12 user nova-compute[71317]:   </features>
Apr 14 04:19:12 user nova-compute[71317]:   <cpu mode='custom' match='exact' check='full'>
Apr 14 04:19:12 user nova-compute[71317]:     <model fallback='forbid'>Nehalem</model>
Apr 14 04:19:12 user nova-compute[71317]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Apr 14 04:19:12 user nova-compute[71317]:     <feature policy='require' name='vme'/>
Apr 14 04:19:12 user nova-compute[71317]:     <feature policy='require' name='x2apic'/>
Apr 14 04:19:12 user nova-compute[71317]:     <feature policy='require' name='hypervisor'/>
Apr 14 04:19:12 user nova-compute[71317]:   </cpu>
Apr 14 04:19:12 user nova-compute[71317]:   <clock offset='utc'>
Apr 14 04:19:12 user nova-compute[71317]:     <timer name='pit' tickpolicy='delay'/>
Apr 14 04:19:12 user nova-compute[71317]:     <timer name='rtc' tickpolicy='catchup'/>
Apr 14 04:19:12 user nova-compute[71317]:     <timer name='hpet' present='no'/>
Apr 14 04:19:12 user nova-compute[71317]:   </clock>
Apr 14 04:19:12 user nova-compute[71317]:   <on_poweroff>destroy</on_poweroff>
Apr 14 04:19:12 user nova-compute[71317]:   <on_reboot>restart</on_reboot>
Apr 14 04:19:12 user nova-compute[71317]:   <on_crash>destroy</on_crash>
Apr 14 04:19:12 user nova-compute[71317]:   <devices>
Apr 14 04:19:12 user nova-compute[71317]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 14 04:19:12 user nova-compute[71317]:     <disk type='file' device='disk'>
Apr 14 04:19:12 user nova-compute[71317]:       <driver name='qemu' type='qcow2' cache='none'/>
Apr 14 04:19:12 user nova-compute[71317]:       <source file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk' index='2'/>
Apr 14 04:19:12 user nova-compute[71317]:       <backingStore type='file' index='3'>
Apr 14 04:19:12 user nova-compute[71317]:         <format type='raw'/>
Apr 14 04:19:12 user nova-compute[71317]:         <source file='/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d'/>
Apr 14 04:19:12 user nova-compute[71317]:         <backingStore/>
Apr 14 04:19:12 user nova-compute[71317]:       </backingStore>
Apr 14 04:19:12 user nova-compute[71317]:       <target dev='vda' bus='virtio'/>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='virtio-disk0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </disk>
Apr 14 04:19:12 user nova-compute[71317]:     <disk type='file' device='cdrom'>
Apr 14 04:19:12 user nova-compute[71317]:       <driver name='qemu' type='raw' cache='none'/>
Apr 14 04:19:12 user nova-compute[71317]:       <source file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk.config' index='1'/>
Apr 14 04:19:12 user nova-compute[71317]:       <backingStore/>
Apr 14 04:19:12 user nova-compute[71317]:       <target dev='hda' bus='ide'/>
Apr 14 04:19:12 user nova-compute[71317]:       <readonly/>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='ide0-0-0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </disk>
Apr 14 04:19:12 user nova-compute[71317]:     <controller type='usb' index='0' model='none'>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='usb'/>
Apr 14 04:19:12 user nova-compute[71317]:     </controller>
Apr 14 04:19:12 user nova-compute[71317]:     <controller type='pci' index='0' model='pci-root'>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='pci.0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </controller>
Apr 14 04:19:12 user nova-compute[71317]:     <controller type='ide' index='0'>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='ide'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Apr 14 04:19:12 user nova-compute[71317]:     </controller>
Apr 14 04:19:12 user nova-compute[71317]:     <interface type='ethernet'>
Apr 14 04:19:12 user nova-compute[71317]:       <mac address='fa:16:3e:58:93:64'/>
Apr 14 04:19:12 user nova-compute[71317]:       <target dev='tap60579bc4-de'/>
Apr 14 04:19:12 user nova-compute[71317]:       <model type='virtio'/>
Apr 14 04:19:12 user nova-compute[71317]:       <mtu size='1442'/>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='net0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </interface>
Apr 14 04:19:12 user nova-compute[71317]:     <serial type='pty'>
Apr 14 04:19:12 user nova-compute[71317]:       <source path='/dev/pts/9'/>
Apr 14 04:19:12 user nova-compute[71317]:       <log file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/console.log' append='off'/>
Apr 14 04:19:12 user nova-compute[71317]:       <target type='isa-serial' port='0'>
Apr 14 04:19:12 user nova-compute[71317]:         <model name='isa-serial'/>
Apr 14 04:19:12 user nova-compute[71317]:       </target>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='serial0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </serial>
Apr 14 04:19:12 user nova-compute[71317]:     <console type='pty' tty='/dev/pts/9'>
Apr 14 04:19:12 user nova-compute[71317]:       <source path='/dev/pts/9'/>
Apr 14 04:19:12 user nova-compute[71317]:       <log file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/console.log' append='off'/>
Apr 14 04:19:12 user nova-compute[71317]:       <target type='serial' port='0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='serial0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </console>
Apr 14 04:19:12 user nova-compute[71317]:     <input type='mouse' bus='ps2'>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='input0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </input>
Apr 14 04:19:12 user nova-compute[71317]:     <input type='keyboard' bus='ps2'>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='input1'/>
Apr 14 04:19:12 user nova-compute[71317]:     </input>
Apr 14 04:19:12 user nova-compute[71317]:     <graphics type='vnc' port='5909' autoport='yes' listen='0.0.0.0'>
Apr 14 04:19:12 user nova-compute[71317]:       <listen type='address' address='0.0.0.0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </graphics>
Apr 14 04:19:12 user nova-compute[71317]:     <audio id='1' type='none'/>
Apr 14 04:19:12 user nova-compute[71317]:     <video>
Apr 14 04:19:12 user nova-compute[71317]:       <model type='virtio' heads='1' primary='yes'/>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='video0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </video>
Apr 14 04:19:12 user nova-compute[71317]:     <memballoon model='virtio'>
Apr 14 04:19:12 user nova-compute[71317]:       <stats period='10'/>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='balloon0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </memballoon>
Apr 14 04:19:12 user nova-compute[71317]:     <rng model='virtio'>
Apr 14 04:19:12 user nova-compute[71317]:       <backend model='random'>/dev/urandom</backend>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='rng0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </rng>
Apr 14 04:19:12 user nova-compute[71317]:   </devices>
Apr 14 04:19:12 user nova-compute[71317]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Apr 14 04:19:12 user nova-compute[71317]:     <label>libvirt-8ed1dea8-e48a-49f7-924e-82cf639a1677</label>
Apr 14 04:19:12 user nova-compute[71317]:     <imagelabel>libvirt-8ed1dea8-e48a-49f7-924e-82cf639a1677</imagelabel>
Apr 14 04:19:12 user nova-compute[71317]:   </seclabel>
Apr 14 04:19:12 user nova-compute[71317]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Apr 14 04:19:12 user nova-compute[71317]:     <label>+64055:+108</label>
Apr 14 04:19:12 user nova-compute[71317]:     <imagelabel>+64055:+108</imagelabel>
Apr 14 04:19:12 user nova-compute[71317]:   </seclabel>
Apr 14 04:19:12 user nova-compute[71317]: </domain>
Apr 14 04:19:12 user nova-compute[71317]:  {{(pid=71317) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:14:36:0b"/><model type="virtio"/><mtu size="1442"/><target dev="tap25048f42-cf"/></interface> {{(pid=71317) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] interface for config: <interface type="ethernet"><mac address="fa:16:3e:14:36:0b"/><model type="virtio"/><mtu size="1442"/><target dev="tap25048f42-cf"/></interface>not found in domain: <domain type='kvm' id='10'>
Apr 14 04:19:12 user nova-compute[71317]:   <name>instance-0000000a</name>
Apr 14 04:19:12 user nova-compute[71317]:   <uuid>8ed1dea8-e48a-49f7-924e-82cf639a1677</uuid>
Apr 14 04:19:12 user nova-compute[71317]:   <metadata>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:19:12 user nova-compute[71317]:   <nova:package version="0.0.0"/>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:name>tempest-device-tagging-server-420804238</nova:name>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:creationTime>2023-04-14 04:19:11</nova:creationTime>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:flavor name="m1.nano">
Apr 14 04:19:12 user nova-compute[71317]:     <nova:memory>128</nova:memory>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:disk>1</nova:disk>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:swap>0</nova:swap>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:vcpus>1</nova:vcpus>
Apr 14 04:19:12 user nova-compute[71317]:   </nova:flavor>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:owner>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:user uuid="0aeeeae7cbc340daa8cd81c1418794e0">tempest-TaggedAttachmentsTest-926384577-project-member</nova:user>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:project uuid="01cefa66350a4432ad71204b39e953be">tempest-TaggedAttachmentsTest-926384577</nova:project>
Apr 14 04:19:12 user nova-compute[71317]:   </nova:owner>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:ports>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:port uuid="60579bc4-de13-48d7-b42d-2b9fc5597e45">
Apr 14 04:19:12 user nova-compute[71317]:       <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 04:19:12 user nova-compute[71317]:     </nova:port>
Apr 14 04:19:12 user nova-compute[71317]:   </nova:ports>
Apr 14 04:19:12 user nova-compute[71317]: </nova:instance>
Apr 14 04:19:12 user nova-compute[71317]:   </metadata>
Apr 14 04:19:12 user nova-compute[71317]:   <memory unit='KiB'>131072</memory>
Apr 14 04:19:12 user nova-compute[71317]:   <currentMemory unit='KiB'>131072</currentMemory>
Apr 14 04:19:12 user nova-compute[71317]:   <vcpu placement='static'>1</vcpu>
Apr 14 04:19:12 user nova-compute[71317]:   <resource>
Apr 14 04:19:12 user nova-compute[71317]:     <partition>/machine</partition>
Apr 14 04:19:12 user nova-compute[71317]:   </resource>
Apr 14 04:19:12 user nova-compute[71317]:   <sysinfo type='smbios'>
Apr 14 04:19:12 user nova-compute[71317]:     <system>
Apr 14 04:19:12 user nova-compute[71317]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Apr 14 04:19:12 user nova-compute[71317]:       <entry name='product'>OpenStack Nova</entry>
Apr 14 04:19:12 user nova-compute[71317]:       <entry name='version'>0.0.0</entry>
Apr 14 04:19:12 user nova-compute[71317]:       <entry name='serial'>8ed1dea8-e48a-49f7-924e-82cf639a1677</entry>
Apr 14 04:19:12 user nova-compute[71317]:       <entry name='uuid'>8ed1dea8-e48a-49f7-924e-82cf639a1677</entry>
Apr 14 04:19:12 user nova-compute[71317]:       <entry name='family'>Virtual Machine</entry>
Apr 14 04:19:12 user nova-compute[71317]:     </system>
Apr 14 04:19:12 user nova-compute[71317]:   </sysinfo>
Apr 14 04:19:12 user nova-compute[71317]:   <os>
Apr 14 04:19:12 user nova-compute[71317]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Apr 14 04:19:12 user nova-compute[71317]:     <boot dev='hd'/>
Apr 14 04:19:12 user nova-compute[71317]:     <smbios mode='sysinfo'/>
Apr 14 04:19:12 user nova-compute[71317]:   </os>
Apr 14 04:19:12 user nova-compute[71317]:   <features>
Apr 14 04:19:12 user nova-compute[71317]:     <acpi/>
Apr 14 04:19:12 user nova-compute[71317]:     <apic/>
Apr 14 04:19:12 user nova-compute[71317]:     <vmcoreinfo state='on'/>
Apr 14 04:19:12 user nova-compute[71317]:   </features>
Apr 14 04:19:12 user nova-compute[71317]:   <cpu mode='custom' match='exact' check='full'>
Apr 14 04:19:12 user nova-compute[71317]:     <model fallback='forbid'>Nehalem</model>
Apr 14 04:19:12 user nova-compute[71317]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Apr 14 04:19:12 user nova-compute[71317]:     <feature policy='require' name='vme'/>
Apr 14 04:19:12 user nova-compute[71317]:     <feature policy='require' name='x2apic'/>
Apr 14 04:19:12 user nova-compute[71317]:     <feature policy='require' name='hypervisor'/>
Apr 14 04:19:12 user nova-compute[71317]:   </cpu>
Apr 14 04:19:12 user nova-compute[71317]:   <clock offset='utc'>
Apr 14 04:19:12 user nova-compute[71317]:     <timer name='pit' tickpolicy='delay'/>
Apr 14 04:19:12 user nova-compute[71317]:     <timer name='rtc' tickpolicy='catchup'/>
Apr 14 04:19:12 user nova-compute[71317]:     <timer name='hpet' present='no'/>
Apr 14 04:19:12 user nova-compute[71317]:   </clock>
Apr 14 04:19:12 user nova-compute[71317]:   <on_poweroff>destroy</on_poweroff>
Apr 14 04:19:12 user nova-compute[71317]:   <on_reboot>restart</on_reboot>
Apr 14 04:19:12 user nova-compute[71317]:   <on_crash>destroy</on_crash>
Apr 14 04:19:12 user nova-compute[71317]:   <devices>
Apr 14 04:19:12 user nova-compute[71317]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 14 04:19:12 user nova-compute[71317]:     <disk type='file' device='disk'>
Apr 14 04:19:12 user nova-compute[71317]:       <driver name='qemu' type='qcow2' cache='none'/>
Apr 14 04:19:12 user nova-compute[71317]:       <source file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk' index='2'/>
Apr 14 04:19:12 user nova-compute[71317]:       <backingStore type='file' index='3'>
Apr 14 04:19:12 user nova-compute[71317]:         <format type='raw'/>
Apr 14 04:19:12 user nova-compute[71317]:         <source file='/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d'/>
Apr 14 04:19:12 user nova-compute[71317]:         <backingStore/>
Apr 14 04:19:12 user nova-compute[71317]:       </backingStore>
Apr 14 04:19:12 user nova-compute[71317]:       <target dev='vda' bus='virtio'/>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='virtio-disk0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </disk>
Apr 14 04:19:12 user nova-compute[71317]:     <disk type='file' device='cdrom'>
Apr 14 04:19:12 user nova-compute[71317]:       <driver name='qemu' type='raw' cache='none'/>
Apr 14 04:19:12 user nova-compute[71317]:       <source file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk.config' index='1'/>
Apr 14 04:19:12 user nova-compute[71317]:       <backingStore/>
Apr 14 04:19:12 user nova-compute[71317]:       <target dev='hda' bus='ide'/>
Apr 14 04:19:12 user nova-compute[71317]:       <readonly/>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='ide0-0-0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </disk>
Apr 14 04:19:12 user nova-compute[71317]:     <controller type='usb' index='0' model='none'>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='usb'/>
Apr 14 04:19:12 user nova-compute[71317]:     </controller>
Apr 14 04:19:12 user nova-compute[71317]:     <controller type='pci' index='0' model='pci-root'>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='pci.0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </controller>
Apr 14 04:19:12 user nova-compute[71317]:     <controller type='ide' index='0'>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='ide'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Apr 14 04:19:12 user nova-compute[71317]:     </controller>
Apr 14 04:19:12 user nova-compute[71317]:     <interface type='ethernet'>
Apr 14 04:19:12 user nova-compute[71317]:       <mac address='fa:16:3e:58:93:64'/>
Apr 14 04:19:12 user nova-compute[71317]:       <target dev='tap60579bc4-de'/>
Apr 14 04:19:12 user nova-compute[71317]:       <model type='virtio'/>
Apr 14 04:19:12 user nova-compute[71317]:       <mtu size='1442'/>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='net0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </interface>
Apr 14 04:19:12 user nova-compute[71317]:     <serial type='pty'>
Apr 14 04:19:12 user nova-compute[71317]:       <source path='/dev/pts/9'/>
Apr 14 04:19:12 user nova-compute[71317]:       <log file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/console.log' append='off'/>
Apr 14 04:19:12 user nova-compute[71317]:       <target type='isa-serial' port='0'>
Apr 14 04:19:12 user nova-compute[71317]:         <model name='isa-serial'/>
Apr 14 04:19:12 user nova-compute[71317]:       </target>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='serial0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </serial>
Apr 14 04:19:12 user nova-compute[71317]:     <console type='pty' tty='/dev/pts/9'>
Apr 14 04:19:12 user nova-compute[71317]:       <source path='/dev/pts/9'/>
Apr 14 04:19:12 user nova-compute[71317]:       <log file='/opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/console.log' append='off'/>
Apr 14 04:19:12 user nova-compute[71317]:       <target type='serial' port='0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='serial0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </console>
Apr 14 04:19:12 user nova-compute[71317]:     <input type='mouse' bus='ps2'>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='input0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </input>
Apr 14 04:19:12 user nova-compute[71317]:     <input type='keyboard' bus='ps2'>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='input1'/>
Apr 14 04:19:12 user nova-compute[71317]:     </input>
Apr 14 04:19:12 user nova-compute[71317]:     <graphics type='vnc' port='5909' autoport='yes' listen='0.0.0.0'>
Apr 14 04:19:12 user nova-compute[71317]:       <listen type='address' address='0.0.0.0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </graphics>
Apr 14 04:19:12 user nova-compute[71317]:     <audio id='1' type='none'/>
Apr 14 04:19:12 user nova-compute[71317]:     <video>
Apr 14 04:19:12 user nova-compute[71317]:       <model type='virtio' heads='1' primary='yes'/>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='video0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </video>
Apr 14 04:19:12 user nova-compute[71317]:     <memballoon model='virtio'>
Apr 14 04:19:12 user nova-compute[71317]:       <stats period='10'/>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='balloon0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </memballoon>
Apr 14 04:19:12 user nova-compute[71317]:     <rng model='virtio'>
Apr 14 04:19:12 user nova-compute[71317]:       <backend model='random'>/dev/urandom</backend>
Apr 14 04:19:12 user nova-compute[71317]:       <alias name='rng0'/>
Apr 14 04:19:12 user nova-compute[71317]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Apr 14 04:19:12 user nova-compute[71317]:     </rng>
Apr 14 04:19:12 user nova-compute[71317]:   </devices>
Apr 14 04:19:12 user nova-compute[71317]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Apr 14 04:19:12 user nova-compute[71317]:     <label>libvirt-8ed1dea8-e48a-49f7-924e-82cf639a1677</label>
Apr 14 04:19:12 user nova-compute[71317]:     <imagelabel>libvirt-8ed1dea8-e48a-49f7-924e-82cf639a1677</imagelabel>
Apr 14 04:19:12 user nova-compute[71317]:   </seclabel>
Apr 14 04:19:12 user nova-compute[71317]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Apr 14 04:19:12 user nova-compute[71317]:     <label>+64055:+108</label>
Apr 14 04:19:12 user nova-compute[71317]:     <imagelabel>+64055:+108</imagelabel>
Apr 14 04:19:12 user nova-compute[71317]:   </seclabel>
Apr 14 04:19:12 user nova-compute[71317]: </domain>
Apr 14 04:19:12 user nova-compute[71317]:  {{(pid=71317) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Apr 14 04:19:12 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Detaching interface fa:16:3e:14:36:0b failed because the device is no longer found on the guest.: nova.exception.DeviceNotFound: Device 'tap25048f42-cf' not found.
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-420804238',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-420804238',id=10,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBvOf9hss+Yqylm6PnX6la1ymm/55r9GRsHJ0NsNe4bgFVG/3wZy3kBc49f/xLbrJ51dzLLHr/o6IQVXcYhlDKHgp4DCKWnVv/anFtE+/Na/vdZgAYvQkLcZYHFyD4dFRQ==',key_name='tempest-keypair-330344245',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata=<?>,migration_context=None,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='01cefa66350a4432ad71204b39e953be',ramdisk_id='',reservation_id='r-p91nk3xp',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-926384577',owner_user_name='tempest-TaggedAttachmentsTest-926384577-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:17:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='0aeeeae7cbc340daa8cd81c1418794e0',uuid=8ed1dea8-e48a-49f7-924e-82cf639a1677,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] Converting VIF {"id": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "address": "fa:16:3e:14:36:0b", "network": {"id": "d28017dc-3976-4caa-a881-077e38a8b0c7", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-227762069", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.130", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap25048f42-cf", "ovs_interfaceid": "25048f42-cfe5-48d1-8620-ac6ac6aa9b82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:14:36:0b,bridge_name='br-int',has_traffic_filtering=True,id=25048f42-cfe5-48d1-8620-ac6ac6aa9b82,network=Network(d28017dc-3976-4caa-a881-077e38a8b0c7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap25048f42-cf') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG os_vif [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:14:36:0b,bridge_name='br-int',has_traffic_filtering=True,id=25048f42-cfe5-48d1-8620-ac6ac6aa9b82,network=Network(d28017dc-3976-4caa-a881-077e38a8b0c7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap25048f42-cf') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap25048f42-cf, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:19:12 user nova-compute[71317]: INFO os_vif [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:14:36:0b,bridge_name='br-int',has_traffic_filtering=True,id=25048f42-cfe5-48d1-8620-ac6ac6aa9b82,network=Network(d28017dc-3976-4caa-a881-077e38a8b0c7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap25048f42-cf')
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [req-963b6ed9-00e8-44b0-986c-fe4ad49ea734 req-29d26773-fa7e-4f5a-98be-beb58f6bff30 service nova] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:19:12 user nova-compute[71317]:   <nova:package version="0.0.0"/>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:name>tempest-device-tagging-server-420804238</nova:name>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:creationTime>2023-04-14 04:19:12</nova:creationTime>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:flavor name="m1.nano">
Apr 14 04:19:12 user nova-compute[71317]:     <nova:memory>128</nova:memory>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:disk>1</nova:disk>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:swap>0</nova:swap>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:vcpus>1</nova:vcpus>
Apr 14 04:19:12 user nova-compute[71317]:   </nova:flavor>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:owner>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:user uuid="0aeeeae7cbc340daa8cd81c1418794e0">tempest-TaggedAttachmentsTest-926384577-project-member</nova:user>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:project uuid="01cefa66350a4432ad71204b39e953be">tempest-TaggedAttachmentsTest-926384577</nova:project>
Apr 14 04:19:12 user nova-compute[71317]:   </nova:owner>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:19:12 user nova-compute[71317]:   <nova:ports>
Apr 14 04:19:12 user nova-compute[71317]:     <nova:port uuid="60579bc4-de13-48d7-b42d-2b9fc5597e45">
Apr 14 04:19:12 user nova-compute[71317]:       <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 04:19:12 user nova-compute[71317]:     </nova:port>
Apr 14 04:19:12 user nova-compute[71317]:   </nova:ports>
Apr 14 04:19:12 user nova-compute[71317]: </nova:instance>
Apr 14 04:19:12 user nova-compute[71317]:  {{(pid=71317) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Apr 14 04:19:12 user nova-compute[71317]: INFO nova.network.neutron [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Port 25048f42-cfe5-48d1-8620-ac6ac6aa9b82 from network info_cache is no longer associated with instance in Neutron. Removing from network info_cache.
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.network.neutron [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Updating instance_info_cache with network_info: [{"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.88", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Releasing lock "refresh_cache-8ed1dea8-e48a-49f7-924e-82cf639a1677" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-12f5a389-dccc-48e0-866c-b307238d063a tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "interface-8ed1dea8-e48a-49f7-924e-82cf639a1677-25048f42-cfe5-48d1-8620-ac6ac6aa9b82" "released" by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" :: held 1.514s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-869462ad-2663-4fde-bf96-da92f31a7e64 req-dfe20067-414f-41e0-93c6-561aaaedd54b service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-vif-unplugged-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-869462ad-2663-4fde-bf96-da92f31a7e64 req-dfe20067-414f-41e0-93c6-561aaaedd54b service nova] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-869462ad-2663-4fde-bf96-da92f31a7e64 req-dfe20067-414f-41e0-93c6-561aaaedd54b service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-869462ad-2663-4fde-bf96-da92f31a7e64 req-dfe20067-414f-41e0-93c6-561aaaedd54b service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-869462ad-2663-4fde-bf96-da92f31a7e64 req-dfe20067-414f-41e0-93c6-561aaaedd54b service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] No waiting events found dispatching network-vif-unplugged-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:12 user nova-compute[71317]: WARNING nova.compute.manager [req-869462ad-2663-4fde-bf96-da92f31a7e64 req-dfe20067-414f-41e0-93c6-561aaaedd54b service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received unexpected event network-vif-unplugged-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 for instance with vm_state active and task_state None.
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-869462ad-2663-4fde-bf96-da92f31a7e64 req-dfe20067-414f-41e0-93c6-561aaaedd54b service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-vif-plugged-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-869462ad-2663-4fde-bf96-da92f31a7e64 req-dfe20067-414f-41e0-93c6-561aaaedd54b service nova] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-869462ad-2663-4fde-bf96-da92f31a7e64 req-dfe20067-414f-41e0-93c6-561aaaedd54b service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-869462ad-2663-4fde-bf96-da92f31a7e64 req-dfe20067-414f-41e0-93c6-561aaaedd54b service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-869462ad-2663-4fde-bf96-da92f31a7e64 req-dfe20067-414f-41e0-93c6-561aaaedd54b service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] No waiting events found dispatching network-vif-plugged-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:12 user nova-compute[71317]: WARNING nova.compute.manager [req-869462ad-2663-4fde-bf96-da92f31a7e64 req-dfe20067-414f-41e0-93c6-561aaaedd54b service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received unexpected event network-vif-plugged-25048f42-cfe5-48d1-8620-ac6ac6aa9b82 for instance with vm_state active and task_state None.
Apr 14 04:19:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] swap_volume: Calling driver volume swap with connection infos: new: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'GcuPjMLZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7', 'attached_at': '', 'detached_at': '', 'volume_id': 'eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370', 'serial': 'eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370'}; old: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PuAKITjr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf'}, 'status': 'reserved', 'instance': 'b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7', 'attached_at': '', 'detached_at': '', 'volume_id': 'f3255f25-0334-46dd-a514-12a56a1deffd', 'serial'} {{(pid=71317) _swap_volume /opt/stack/nova/nova/compute/manager.py:7656}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'GcuPjMLZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:16 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:16 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:16 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f17b41d7-2e04-4cce-be75-9588e623529c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:19:16 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e95abd7-2132-4810-b12f-087d9586193e]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370] added
Apr 14 04:19:16 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d9d4060-5b6f-4489-9b40-09c562276009]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3af3bf5a-6a1e-4c1f-803f-e7c44e94de7e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v GcuPjMLZ {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v GcuPjMLZ" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[667da00f-92a1-42ae-a958-2523c684f531]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'GcuPjMLZ'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7913bff8-988b-4246-8ed9-5075a737bc87]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[32341604-5cb3-4a6a-995e-6586e8a76e48]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:16 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:16 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --login" returned: 0 in 0.073s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[07f38440-7d1e-4880-860c-300fa0ebf090]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370, portal: 172.16.0.220,3260]
Apr 14 04:19:16 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:16 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[85e3b045-0c1b-4bea-b807-c189ac47e98f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[da772da0-98e4-4ec2-9cca-7214b3d930d4]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:16 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:16 user nova-compute[71317]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:16 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.286s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] HCTL ('38', '-', '-', 0) found on session 10 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Scanning host 38 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b866c02b-fe29-4e23-94af-a0d617fa9cdb]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Searching for a device in session 10 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:19:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Connected to sdg using {'target_iqn': 'iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'GcuPjMLZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:19:17 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.319s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:17 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] <== connect_volume: return (1321ms) {'type': 'block', 'scsi_wwn': '26238353462303336', 'path': '/dev/sdg'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26238353462303336', 'path': '/dev/sdg'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:19:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lazy-loading 'flavor' on Instance uuid b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 7340032 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 105906176 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 277872640 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 467664896 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:20 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 639631360 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:20 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 812646400 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 978321408 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 1073741824 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PuAKITjr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:21 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[54464e12-216e-4d4e-9f78-459cbcc13a1d]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:19:21 user nova-compute[71317]: No targets found.
Apr 14 04:19:21 user nova-compute[71317]: iSNS:
Apr 14 04:19:21 user nova-compute[71317]: No targets found.
Apr 14 04:19:21 user nova-compute[71317]: STATIC:
Apr 14 04:19:21 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282
Apr 14 04:19:21 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:21 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:21 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd
Apr 14 04:19:21 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:21 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:21 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370
Apr 14 04:19:21 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:21 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:21 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b
Apr 14 04:19:21 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:21 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:21 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:19:21 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:21 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:21 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3
Apr 14 04:19:21 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:21 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:21 user nova-compute[71317]: FIRMWARE:
Apr 14 04:19:21 user nova-compute[71317]: No targets found.
Apr 14 04:19:21 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:19:21 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:19:21 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c7deb80-1858-4691-aa3a-b471a4fc1605]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a90009df-5e60-470f-b3e1-de6e3da7a353]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)
Apr 14 04:19:21 user nova-compute[71317]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:21 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:21 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:21 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:21 user nova-compute[71317]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:21 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)
Apr 14 04:19:21 user nova-compute[71317]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:21 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:21 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:21 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:21 user nova-compute[71317]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:21 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84b4217520>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd'): ({'sdf'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Removing single pathed devices sdf {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7637ac1f-b1e2-44a1-8053-2b2d064f6857]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[530282ae-3214-4f01-8761-88019a80a1f2]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Flushing IO for device /dev/sdf {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4f18bbe-bced-41f1-b531-d8740dd9decf]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:19:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.120s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6600e379-8102-4153-89f4-d5b774e9b9f6]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Checking to see if SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a6fb2b4-e582-4d32-9cf1-fa891b46bcf8]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --logout" returned: 0 in 0.068s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fc199e0-b5ce-46b1-9d9b-92af5174eb5c]: (4, ('Logging out of session [sid: 9, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260]\nLogout of [sid: 9, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--logout',): stdout=Logging out of session [sid: 9, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260]
Apr 14 04:19:22 user nova-compute[71317]: Logout of [sid: 9, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:22 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op delete" returned: 0 in 0.300s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[afaaef8c-4505-4e75-b5c1-5b1a2dbc9e80]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.661s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] <== disconnect_volume: return (662ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] swap_volume: Driver volume swap returned, new connection_info is now : {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'GcuPjMLZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg'}, 'status': 'reserved', 'instance': 'b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7', 'attached_at': '', 'detached_at': '', 'volume_id': 'eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370', 'serial'} {{(pid=71317) _swap_volume /opt/stack/nova/nova/compute/manager.py:7665}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] swap_volume: removing Cinder connection for volume f3255f25-0334-46dd-a514-12a56a1deffd {{(pid=71317) _swap_volume /opt/stack/nova/nova/compute/manager.py:7710}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updating instance_info_cache with network_info: [{"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:19:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:19:23 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:19:23 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:19:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] swap_volume: Updating volume f3255f25-0334-46dd-a514-12a56a1deffd BDM record with {'connection_info': '{"driver_volume_type": "iscsi", "data": {"target_iqn": "iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370", "target_portal": "172.16.0.220:3260", "target_discovered": false, "auth_method": "CHAP", "auth_username": "GcuPjMLZ", "auth_password": "MgPp9wXI7yqnUY", "target_lun": 0, "qos_specs": null, "access_mode": "rw", "encrypted": false, "cacheable": false, "device_path": "/dev/sdg", "volume_id": "eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370"}, "status": "reserved", "instance": "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7", "attached_at": "", "detached_at": "", "volume_id": "eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370", "serial": "eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370"}', 'source_type': 'volume', 'destination_type': 'volume', 'snapshot_id': None, 'volume_id': 'eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370', 'no_device': None, 'attachment_id': 'a180602f-c69a-4f73-9bb5-210d716aec55'} {{(pid=71317) _do_swap_volume /opt/stack/nova/nova/compute/manager.py:7858}}
Apr 14 04:19:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-18d7ebe5-0721-40e7-b84a-27f3b8debe41 tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" "released" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: held 11.721s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Acquiring lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" acquired by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:23 user nova-compute[71317]: DEBUG os_brick.utils [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a00e8196-363b-407d-bdc9-45ecbd025b53]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9676e86a-74c1-45bb-b91a-a37c86c4cf89]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681445949.0242338, e5344b0d-2d73-40c7-8ac3-535396e985a4 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:19:24 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] VM Stopped (Lifecycle Event)
Apr 14 04:19:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[25d39bde-9363-4f62-80a8-fa65335bb15e]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1c19f73-3e18-4abf-b9bd-1b7c2e356a84]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG nova.compute.manager [None req-11e32f4c-a003-42c2-9cb6-e94719ef97db None None] [instance: e5344b0d-2d73-40c7-8ac3-535396e985a4] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:19:24 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:19:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee8838c5-55ee-4dc3-9daa-74e37844e3c8]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:19:24 user nova-compute[71317]: DEBUG os_brick.utils [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] <== get_connector_properties: return (114ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:24 user nova-compute[71317]: INFO nova.compute.manager [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Swapping volume eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 for f3255f25-0334-46dd-a514-12a56a1deffd
Apr 14 04:19:25 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:19:25 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] swap_volume: Calling driver volume swap with connection infos: new: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PuAKITjr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7', 'attached_at': '', 'detached_at': '', 'volume_id': 'f3255f25-0334-46dd-a514-12a56a1deffd', 'serial': 'f3255f25-0334-46dd-a514-12a56a1deffd'}; old: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'GcuPjMLZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg', 'volume_id': 'eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370'}, 'status': 'reserved', 'instance': 'b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7', 'attached_at': '', 'detached_at': '', 'volume_id': 'eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370', 'serial'} {{(pid=71317) _swap_volume /opt/stack/nova/nova/compute/manager.py:7656}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PuAKITjr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:28 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:28 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:28 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d8b90bb-588a-4e3d-893a-332d553e2b33]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:19:28 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdg (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cfb99679-680a-4302-ab42-edfb985c91a8]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd] added
Apr 14 04:19:28 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[77d59574-3c70-4dbe-b72a-dd3c466ed8ac]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[af6c8571-1119-43fb-b82a-6872cb7dc73e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v PuAKITjr {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v PuAKITjr" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b911c166-1df3-4131-a3c0-09472cf1dbdd]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'PuAKITjr'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e631f6d-c9ea-43c2-8da8-ce66b28a738b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ed66797-7e8e-48b0-b2b3-dd63b9e89680]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --login" returned: 0 in 0.074s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b29492c-6191-47ec-b671-0ea0fb76bac1]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260]
Apr 14 04:19:28 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:28 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[489c2e37-ed5d-49b6-b02e-f950ce0b60ef]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a291e68f-db24-4dee-bd89-196136112b17]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:28 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.356s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] HCTL ('37', '-', '-', 0) found on session 11 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Scanning host 37 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:19:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e47a093b-178f-4d4c-97dd-70af17dca437]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Searching for a device in session 11 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PuAKITjr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:19:29 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vdc) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.703s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] <== connect_volume: return (1706ms) {'type': 'block', 'scsi_wwn': '23330303765366434', 'path': '/dev/sdf'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23330303765366434', 'path': '/dev/sdf'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:19:30 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:19:30 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7354MB free_disk=26.48165512084961GB free_vcpus=1 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.objects.instance [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lazy-loading 'flavor' on Instance uuid b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 9b532a44-20b7-4590-babe-684c49598a62 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance da3b4651-aaa0-430d-85b8-c15de386d08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 8ed1dea8-e48a-49f7-924e-82cf639a1677 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 55ba1cae-2b2f-43de-86db-996a8791fb02 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 59e547de-a74b-4ccd-90de-e7b61cba6964 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 313835fa-6405-41ba-bc89-c4bfc119326a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 12 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=2048MB phys_disk=40GB used_disk=11GB total_vcpus=12 used_vcpus=12 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 0 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:19:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.584s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 93323264 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 218103808 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 391118848 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 522190848 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 642777088 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 761266176 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:34 user nova-compute[71317]: DEBUG nova.compute.manager [req-b5ecd7cb-ee77-4e58-9f98-3ead13b7939c req-71bd4c72-1553-4e76-9325-130b542d06c0 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-changed-eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:34 user nova-compute[71317]: DEBUG nova.compute.manager [req-b5ecd7cb-ee77-4e58-9f98-3ead13b7939c req-71bd4c72-1553-4e76-9325-130b542d06c0 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing instance network info cache due to event network-changed-eea29fce-f6e3-4715-9a46-2a638462c6f8. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:19:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b5ecd7cb-ee77-4e58-9f98-3ead13b7939c req-71bd4c72-1553-4e76-9325-130b542d06c0 service nova] Acquiring lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:19:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b5ecd7cb-ee77-4e58-9f98-3ead13b7939c req-71bd4c72-1553-4e76-9325-130b542d06c0 service nova] Acquired lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:19:34 user nova-compute[71317]: DEBUG nova.network.neutron [req-b5ecd7cb-ee77-4e58-9f98-3ead13b7939c req-71bd4c72-1553-4e76-9325-130b542d06c0 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Refreshing network info cache for port eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:19:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 916455424 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 1069547520 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG nova.network.neutron [req-b5ecd7cb-ee77-4e58-9f98-3ead13b7939c req-71bd4c72-1553-4e76-9325-130b542d06c0 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updated VIF entry in instance network info cache for port eea29fce-f6e3-4715-9a46-2a638462c6f8. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG nova.network.neutron [req-b5ecd7cb-ee77-4e58-9f98-3ead13b7939c req-71bd4c72-1553-4e76-9325-130b542d06c0 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating instance_info_cache with network_info: [{"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b5ecd7cb-ee77-4e58-9f98-3ead13b7939c req-71bd4c72-1553-4e76-9325-130b542d06c0 service nova] Releasing lock "refresh_cache-55ba1cae-2b2f-43de-86db-996a8791fb02" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] COPY block job progress, current cursor: 1073741824 final cursor: 1073741824 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'GcuPjMLZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg', 'volume_id': 'eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:35 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cf5a29e5-7699-4402-9f68-5fd1737a81e0]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:19:35 user nova-compute[71317]: No targets found.
Apr 14 04:19:35 user nova-compute[71317]: iSNS:
Apr 14 04:19:35 user nova-compute[71317]: No targets found.
Apr 14 04:19:35 user nova-compute[71317]: STATIC:
Apr 14 04:19:35 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282
Apr 14 04:19:35 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:35 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:35 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd
Apr 14 04:19:35 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:35 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:35 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370
Apr 14 04:19:35 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:35 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:35 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b
Apr 14 04:19:35 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:35 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:35 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:19:35 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:35 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:35 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3
Apr 14 04:19:35 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:35 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:35 user nova-compute[71317]: FIRMWARE:
Apr 14 04:19:35 user nova-compute[71317]: No targets found.
Apr 14 04:19:35 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:19:35 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:19:35 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9356dfa-144b-4556-8789-cd9a10b5cdfd]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7fe65f7-f0b4-4b51-9e96-64ec885f11a2]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)
Apr 14 04:19:35 user nova-compute[71317]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:35 user nova-compute[71317]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:35 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:35 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:35 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:35 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 (non-flash)
Apr 14 04:19:35 user nova-compute[71317]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:35 user nova-compute[71317]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:35 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:35 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:35 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:35 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d1975120>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370'): ({'sdg'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Removing single pathed devices sdg {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[950308de-b4a6-44c4-8e35-f43290ae6ebf]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7cb25282-073a-4a51-8eb6-5b8fc7b23cad]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Flushing IO for device /dev/sdg {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3eef29a-fe94-4578-ba01-2c4ca0b7db12]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.043s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ec313c2-a3e0-47ae-838d-3e2b26070f3a]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Checking to see if SCSI volumes sdg have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] SCSI volumes sdg have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[001a2081-dd6d-4219-8183-a42b202a947e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --logout" returned: 0 in 0.074s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e721acb4-576b-4280-8a0a-d9ba75c636d1]: (4, ('Logging out of session [sid: 10, target: iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370, portal: 172.16.0.220,3260]\nLogout of [sid: 10, target: iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--logout',): stdout=Logging out of session [sid: 10, target: iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370, portal: 172.16.0.220,3260]
Apr 14 04:19:35 user nova-compute[71317]: Logout of [sid: 10, target: iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:35 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.036s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f96947b-ee42-4778-b3b0-ab595a960d66]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.363s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] <== disconnect_volume: return (364ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] swap_volume: Driver volume swap returned, new connection_info is now : {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PuAKITjr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf'}, 'status': 'reserved', 'instance': 'b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7', 'attached_at': '', 'detached_at': '', 'volume_id': 'f3255f25-0334-46dd-a514-12a56a1deffd', 'serial'} {{(pid=71317) _swap_volume /opt/stack/nova/nova/compute/manager.py:7665}}
Apr 14 04:19:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] swap_volume: removing Cinder connection for volume eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 {{(pid=71317) _swap_volume /opt/stack/nova/nova/compute/manager.py:7710}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:36 user nova-compute[71317]: INFO nova.compute.manager [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Terminating instance
Apr 14 04:19:36 user nova-compute[71317]: DEBUG nova.compute.manager [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG nova.compute.manager [req-a73bf814-3a92-4ff2-9007-37bee9b7a6bf req-a28698ae-9bf6-433f-97f1-676fc07517aa service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a73bf814-3a92-4ff2-9007-37bee9b7a6bf req-a28698ae-9bf6-433f-97f1-676fc07517aa service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a73bf814-3a92-4ff2-9007-37bee9b7a6bf req-a28698ae-9bf6-433f-97f1-676fc07517aa service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a73bf814-3a92-4ff2-9007-37bee9b7a6bf req-a28698ae-9bf6-433f-97f1-676fc07517aa service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG nova.compute.manager [req-a73bf814-3a92-4ff2-9007-37bee9b7a6bf req-a28698ae-9bf6-433f-97f1-676fc07517aa service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-unplugged-eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG nova.compute.manager [req-a73bf814-3a92-4ff2-9007-37bee9b7a6bf req-a28698ae-9bf6-433f-97f1-676fc07517aa service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-eea29fce-f6e3-4715-9a46-2a638462c6f8 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG nova.compute.manager [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] swap_volume: Updating volume eb2fcdfc-806a-4eb7-8ed3-bb78c58cc370 BDM record with {'connection_info': '{"driver_volume_type": "iscsi", "data": {"target_iqn": "iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd", "target_portal": "172.16.0.220:3260", "target_discovered": false, "auth_method": "CHAP", "auth_username": "PuAKITjr", "auth_password": "51syx2hVFSmOTK", "target_lun": 0, "qos_specs": null, "access_mode": "rw", "encrypted": false, "cacheable": false, "device_path": "/dev/sdf", "volume_id": "f3255f25-0334-46dd-a514-12a56a1deffd"}, "status": "reserved", "instance": "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7", "attached_at": "", "detached_at": "", "volume_id": "f3255f25-0334-46dd-a514-12a56a1deffd", "serial": "f3255f25-0334-46dd-a514-12a56a1deffd"}', 'source_type': 'volume', 'destination_type': 'volume', 'snapshot_id': None, 'volume_id': 'f3255f25-0334-46dd-a514-12a56a1deffd', 'no_device': None, 'attachment_id': '390b336c-ff30-43de-b2e8-acbd0f195610'} {{(pid=71317) _do_swap_volume /opt/stack/nova/nova/compute/manager.py:7858}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b832fbb5-91ab-4a5e-a1b5-b07052007feb tempest-TestVolumeSwap-117798025 tempest-TestVolumeSwap-117798025-project-admin] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" "released" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: held 12.944s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-d8a933fd-578b-4393-b4cb-1a7343b4c27e req-84482fc8-025b-4676-8102-3388278d4d9d service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d8a933fd-578b-4393-b4cb-1a7343b4c27e req-84482fc8-025b-4676-8102-3388278d4d9d service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d8a933fd-578b-4393-b4cb-1a7343b4c27e req-84482fc8-025b-4676-8102-3388278d4d9d service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d8a933fd-578b-4393-b4cb-1a7343b4c27e req-84482fc8-025b-4676-8102-3388278d4d9d service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-d8a933fd-578b-4393-b4cb-1a7343b4c27e req-84482fc8-025b-4676-8102-3388278d4d9d service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-unplugged-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-d8a933fd-578b-4393-b4cb-1a7343b4c27e req-84482fc8-025b-4676-8102-3388278d4d9d service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:37 user nova-compute[71317]: INFO nova.compute.manager [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Detaching volume f3255f25-0334-46dd-a514-12a56a1deffd
Apr 14 04:19:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:37 user nova-compute[71317]: INFO nova.virt.block_device [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Attempting to driver detach volume f3255f25-0334-46dd-a514-12a56a1deffd from mountpoint /dev/vdb
Apr 14 04:19:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Attempting to detach device vdb from instance b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:19:37 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:19:37 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:19:37 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:19:37 user nova-compute[71317]:   <serial>f3255f25-0334-46dd-a514-12a56a1deffd</serial>
Apr 14 04:19:37 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:19:37 user nova-compute[71317]: </disk>
Apr 14 04:19:37 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:19:37 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Successfully detached device vdb from instance b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 from the persistent domain config.
Apr 14 04:19:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:19:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:19:37 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:19:37 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:19:37 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:19:37 user nova-compute[71317]:   <serial>f3255f25-0334-46dd-a514-12a56a1deffd</serial>
Apr 14 04:19:37 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:19:37 user nova-compute[71317]: </disk>
Apr 14 04:19:37 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681445978.4308846, b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:19:38 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Successfully detached device vdb from instance b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 from the live domain config.
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PuAKITjr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf', 'volume_id': 'f3255f25-0334-46dd-a514-12a56a1deffd': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:38 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:38 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Instance destroyed successfully.
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.objects.instance [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lazy-loading 'resources' on Instance uuid 55ba1cae-2b2f-43de-86db-996a8791fb02 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:18:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "address": "fa:16:3e:d1:8d:17", "network": {"id": "a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-198146007-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeea29fce-f6", "ovs_interfaceid": "eea29fce-f6e3-4715-9a46-2a638462c6f8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d1:8d:17,bridge_name='br-int',has_traffic_filtering=True,id=eea29fce-f6e3-4715-9a46-2a638462c6f8,network=Network(a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeea29fce-f6') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d1:8d:17,bridge_name='br-int',has_traffic_filtering=True,id=eea29fce-f6e3-4715-9a46-2a638462c6f8,network=Network(a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeea29fce-f6') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[82090095-769c-4b80-80d6-184090f629f9]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:19:38 user nova-compute[71317]: No targets found.
Apr 14 04:19:38 user nova-compute[71317]: iSNS:
Apr 14 04:19:38 user nova-compute[71317]: No targets found.
Apr 14 04:19:38 user nova-compute[71317]: STATIC:
Apr 14 04:19:38 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282
Apr 14 04:19:38 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:38 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:38 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd
Apr 14 04:19:38 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:38 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:38 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b
Apr 14 04:19:38 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:38 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:38 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:19:38 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:38 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:38 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3
Apr 14 04:19:38 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:38 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:38 user nova-compute[71317]: FIRMWARE:
Apr 14 04:19:38 user nova-compute[71317]: No targets found.
Apr 14 04:19:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:19:38 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:19:38 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapeea29fce-f6, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[53b11c01-9326-43c5-9282-b0622c42afdd]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa4e4eb3-eb44-40d4-ba53-db29c27847f3]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:38 user nova-compute[71317]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:38 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:38 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:38 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd (non-flash)
Apr 14 04:19:38 user nova-compute[71317]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:38 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:38 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:38 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d180a710>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd'): ({'sdf'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Removing single pathed devices sdf {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: INFO os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d1:8d:17,bridge_name='br-int',has_traffic_filtering=True,id=eea29fce-f6e3-4715-9a46-2a638462c6f8,network=Network(a1bf1e4d-8f7d-47bf-89a9-85f81fe3b428),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeea29fce-f6')
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:18:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b3:13:af,bridge_name='br-int',has_traffic_filtering=True,id=8c995dcb-b43b-4bdd-b631-e3b6104d49d0,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8c995dcb-b4') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b3:13:af,bridge_name='br-int',has_traffic_filtering=True,id=8c995dcb-b43b-4bdd-b631-e3b6104d49d0,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8c995dcb-b4') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ffdfa7cc-f538-44a8-80eb-aa37c442053c]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8c995dcb-b4, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a00efe2-bc08-477c-a43a-027842db1641]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Flushing IO for device /dev/sdf {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccfc1d42-9654-44d3-9399-d12134525f28]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: INFO os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b3:13:af,bridge_name='br-int',has_traffic_filtering=True,id=8c995dcb-b43b-4bdd-b631-e3b6104d49d0,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8c995dcb-b4')
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:18:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:86:19:cc,bridge_name='br-int',has_traffic_filtering=True,id=c9994009-b14a-4ff4-ac36-f02b0feccfb6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapc9994009-b1') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:86:19:cc,bridge_name='br-int',has_traffic_filtering=True,id=c9994009-b14a-4ff4-ac36-f02b0feccfb6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapc9994009-b1') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc9994009-b1, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.049s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[07fe70f5-0fdb-4aaa-8d0b-856df39d32b2]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f7d191d-5117-4f55-982d-489c04ee9d58]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:38 user nova-compute[71317]: INFO os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:86:19:cc,bridge_name='br-int',has_traffic_filtering=True,id=c9994009-b14a-4ff4-ac36-f02b0feccfb6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapc9994009-b1')
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:18:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:48:3e:f5,bridge_name='br-int',has_traffic_filtering=True,id=11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap11fd6438-8a') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:48:3e:f5,bridge_name='br-int',has_traffic_filtering=True,id=11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap11fd6438-8a') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap11fd6438-8a, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: INFO os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:48:3e:f5,bridge_name='br-int',has_traffic_filtering=True,id=11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap11fd6438-8a')
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:18:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:95:73:7e,bridge_name='br-int',has_traffic_filtering=True,id=8257c16c-6115-4b67-a739-91cb1d5e8bc6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8257c16c-61') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:95:73:7e,bridge_name='br-int',has_traffic_filtering=True,id=8257c16c-6115-4b67-a739-91cb1d5e8bc6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8257c16c-61') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --logout" returned: 0 in 0.159s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[adc5c8d5-5452-4763-b2c8-f2505a51f027]: (4, ('Logging out of session [sid: 11, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260]\nLogout of [sid: 11, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8257c16c-61, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 11, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260]
Apr 14 04:19:38 user nova-compute[71317]: Logout of [sid: 11, target: iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:38 user nova-compute[71317]: WARNING nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-eea29fce-f6e3-4715-9a46-2a638462c6f8 for instance with vm_state active and task_state deleting.
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-unplugged-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:38 user nova-compute[71317]: WARNING nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe for instance with vm_state active and task_state deleting.
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-unplugged-8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-8257c16c-6115-4b67-a739-91cb1d5e8bc6 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.012s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:38 user nova-compute[71317]: WARNING nova.compute.manager [req-e5f02f87-45e2-49ee-9c34-a85886a450bf req-ce94e23a-7559-44c3-b62c-7032a8b9c804 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-8257c16c-6115-4b67-a739-91cb1d5e8bc6 for instance with vm_state active and task_state deleting.
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f3255f25-0334-46dd-a514-12a56a1deffd -p 172.16.0.220:3260 --op delete" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[34842485-2501-45a1-a8f3-aaa6adf2fac8]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.473s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] <== disconnect_volume: return (476ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:19:38 user nova-compute[71317]: INFO os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:95:73:7e,bridge_name='br-int',has_traffic_filtering=True,id=8257c16c-6115-4b67-a739-91cb1d5e8bc6,network=Network(ca386160-6d9c-4824-a9c3-a50508890008),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8257c16c-61')
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:18:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9c:4f:8e,bridge_name='br-int',has_traffic_filtering=True,id=07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07a4c8ef-59') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9c:4f:8e,bridge_name='br-int',has_traffic_filtering=True,id=07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07a4c8ef-59') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap07a4c8ef-59, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: INFO os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9c:4f:8e,bridge_name='br-int',has_traffic_filtering=True,id=07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07a4c8ef-59')
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-889777707',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-889777707',id=12,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMCXMmSyKLZyq4lyKnG7khEHSc5g5qMJ/F8GRCn7oNYqNxpgFzEfwdZtFyeQ/uI5EAoHbRxxVEOfw5Nqy2Ndi9EyNPabCR+EMVPuahnBulje1M1+So1WNA2mqeYZNAT8kg==',key_name='tempest-keypair-475824842',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='225e61ca983d4f27adc61ad33c81fce4',ramdisk_id='',reservation_id='r-f39xf1y5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-581315356',owner_user_name='tempest-TaggedBootDevicesTest_v242-581315356-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:18:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='b848680106b74275a947f316ce6311fd',uuid=55ba1cae-2b2f-43de-86db-996a8791fb02,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converting VIF {"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:53:d2:74,bridge_name='br-int',has_traffic_filtering=True,id=512baf47-b5bf-4cf0-8300-f0e0167fc70e,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap512baf47-b5') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:53:d2:74,bridge_name='br-int',has_traffic_filtering=True,id=512baf47-b5bf-4cf0-8300-f0e0167fc70e,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap512baf47-b5') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap512baf47-b5, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:39 user nova-compute[71317]: INFO os_vif [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:53:d2:74,bridge_name='br-int',has_traffic_filtering=True,id=512baf47-b5bf-4cf0-8300-f0e0167fc70e,network=Network(a05db065-8e40-45f0-b7b3-49aa6fa45eed),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap512baf47-b5')
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ZFpuzxGw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:39 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9fef4b93-381f-4c47-93b6-63e0edc04545]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:19:39 user nova-compute[71317]: No targets found.
Apr 14 04:19:39 user nova-compute[71317]: iSNS:
Apr 14 04:19:39 user nova-compute[71317]: No targets found.
Apr 14 04:19:39 user nova-compute[71317]: STATIC:
Apr 14 04:19:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282
Apr 14 04:19:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b
Apr 14 04:19:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:19:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3
Apr 14 04:19:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:39 user nova-compute[71317]: FIRMWARE:
Apr 14 04:19:39 user nova-compute[71317]: No targets found.
Apr 14 04:19:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:19:39 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:19:39 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b2c5b82-4405-47cf-8d64-051ee5028bcc]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:39 user nova-compute[71317]: WARNING nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-8c995dcb-b43b-4bdd-b631-e3b6104d49d0 for instance with vm_state active and task_state deleting.
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-c9994009-b14a-4ff4-ac36-f02b0feccfb6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-unplugged-c9994009-b14a-4ff4-ac36-f02b0feccfb6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-c9994009-b14a-4ff4-ac36-f02b0feccfb6 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-c9994009-b14a-4ff4-ac36-f02b0feccfb6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-c9994009-b14a-4ff4-ac36-f02b0feccfb6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:39 user nova-compute[71317]: WARNING nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-c9994009-b14a-4ff4-ac36-f02b0feccfb6 for instance with vm_state active and task_state deleting.
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-unplugged-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:39 user nova-compute[71317]: WARNING nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 for instance with vm_state active and task_state deleting.
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-unplugged-512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-unplugged-512baf47-b5bf-4cf0-8300-f0e0167fc70e for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-plugged-512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Acquiring lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] No waiting events found dispatching network-vif-plugged-512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:19:39 user nova-compute[71317]: WARNING nova.compute.manager [req-2e51c944-ba82-4528-90e1-94b23c58a4fb req-be5a2f41-fe07-4462-9e56-2caad3b93add service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received unexpected event network-vif-plugged-512baf47-b5bf-4cf0-8300-f0e0167fc70e for instance with vm_state active and task_state deleting.
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5262851-8132-4d32-8d83-e1f55a23a95b]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:39 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:39 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c41e4310>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3'): ({'sdb'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Removing single pathed devices sdb {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed09e68c-341d-4891-8d2c-e4414228d1d0]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca053101-94d9-4b99-a3f2-e9a2abf69141]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Flushing IO for device /dev/sdb {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cdf18ba4-d200-4e07-91cd-343ad7807063]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.043s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1df262b7-b97e-43d7-af34-9b949d1bc78c]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcb371cf-44f3-4926-8fbc-b306846625c4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --logout" returned: 0 in 0.056s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[369a4ed2-1b97-4578-ba17-b3db1a7fad18]: (4, ('Logging out of session [sid: 2, target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3, portal: 172.16.0.220,3260]\nLogout of [sid: 2, target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 2, target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3, portal: 172.16.0.220,3260]
Apr 14 04:19:39 user nova-compute[71317]: Logout of [sid: 2, target: iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:31382528-1966-4f19-8a4a-94983a6314c3 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7523c9e1-44e5-4931-9bd5-f5c42542fe05]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.375s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] <== disconnect_volume: return (376ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nQRoZmkl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:39 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[83c7fdbc-5b40-49b6-8d95-19fda1c50aa8]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:19:39 user nova-compute[71317]: No targets found.
Apr 14 04:19:39 user nova-compute[71317]: iSNS:
Apr 14 04:19:39 user nova-compute[71317]: No targets found.
Apr 14 04:19:39 user nova-compute[71317]: STATIC:
Apr 14 04:19:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282
Apr 14 04:19:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b
Apr 14 04:19:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:19:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:39 user nova-compute[71317]: FIRMWARE:
Apr 14 04:19:39 user nova-compute[71317]: No targets found.
Apr 14 04:19:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:19:39 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:19:39 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b89f8ef-a774-425e-929d-8d1cb95cceae]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[271b9c43-567c-4002-b0b1-75cdd9189752]: (4, ('tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:39 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsi session list stdout=tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b (non-flash)
Apr 14 04:19:39 user nova-compute[71317]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d1974820>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[23d734e9-6ca3-4617-8e86-90bde84dc55e]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e44d46c2-3984-4501-aae6-76452fbc5a66]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.012s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd89433a-bedd-439b-b44f-b858b78595c8]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.038s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[369ae20f-7b3a-4a7f-957c-506e739ccaef]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[42580f3f-bbda-43c6-acca-a5d1720ac2b4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --logout" returned: 0 in 0.061s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7760f5e-8176-4606-8f1b-43510f8dbcec]: (4, ('Logging out of session [sid: 3, target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b, portal: 172.16.0.220,3260]\nLogout of [sid: 3, target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 3, target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b, portal: 172.16.0.220,3260]
Apr 14 04:19:39 user nova-compute[71317]: Logout of [sid: 3, target: iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:18016742-a15a-4e85-9fb9-4cbfd819316b -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a9f9772-bfd7-4eab-ba78-90fa63adfd48]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.304s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] <== disconnect_volume: return (306ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HjUKqOuQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:39 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bf317a4-2cc9-4ba2-b924-e3724144ee3b]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:19:39 user nova-compute[71317]: No targets found.
Apr 14 04:19:39 user nova-compute[71317]: iSNS:
Apr 14 04:19:39 user nova-compute[71317]: No targets found.
Apr 14 04:19:39 user nova-compute[71317]: STATIC:
Apr 14 04:19:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282
Apr 14 04:19:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:19:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:19:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:19:39 user nova-compute[71317]: FIRMWARE:
Apr 14 04:19:39 user nova-compute[71317]: No targets found.
Apr 14 04:19:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:19:39 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:19:39 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[265daa1f-c52c-447e-ba32-62e5eba39e77]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[49f38a83-50ce-422b-a61d-fd2fbcdb2479]: (4, ('tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsi session list stdout=tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d1974820>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282'): ({'sdd'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Removing single pathed devices sdd {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG nova.objects.instance [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lazy-loading 'flavor' on Instance uuid b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c2ecb2d-51ac-4e19-af30-9e2eaf9680a4]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0153607a-2f0b-4273-bfbe-05f1abe175eb]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Flushing IO for device /dev/sdd {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3bafef37-c05b-4e2b-8f75-eb55e5b2b9be]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b3742b90-6cb5-448d-a7b9-50727360ccc6 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.193s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.045s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[da27aed4-307a-4413-abd9-6c2d7b8de63f]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b49b0e77-d5d8-42bd-ba37-ab3751251868]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --logout" returned: 0 in 0.047s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cefc5d22-b176-4ab5-b346-9b2b64d4c2ca]: (4, ('Logging out of session [sid: 4, target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282, portal: 172.16.0.220,3260]\nLogout of [sid: 4, target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 4, target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282, portal: 172.16.0.220,3260]
Apr 14 04:19:39 user nova-compute[71317]: Logout of [sid: 4, target: iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c8315093-aba5-4adf-8381-c59806b42282 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[00038b4a-0e31-4dbe-9abc-49d9454e72be]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.298s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] <== disconnect_volume: return (298ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:19:40 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Deleting instance files /opt/stack/data/nova/instances/55ba1cae-2b2f-43de-86db-996a8791fb02_del
Apr 14 04:19:40 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Deletion of /opt/stack/data/nova/instances/55ba1cae-2b2f-43de-86db-996a8791fb02_del complete
Apr 14 04:19:40 user nova-compute[71317]: INFO nova.compute.manager [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Took 3.63 seconds to destroy the instance on the hypervisor.
Apr 14 04:19:40 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:19:40 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:19:40 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:19:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-410ee83a-1aaa-4a13-8cac-4a563317a793 req-4e73289a-bc89-4d56-b754-801eab0e7a77 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-deleted-eea29fce-f6e3-4715-9a46-2a638462c6f8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:42 user nova-compute[71317]: INFO nova.compute.manager [req-410ee83a-1aaa-4a13-8cac-4a563317a793 req-4e73289a-bc89-4d56-b754-801eab0e7a77 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Neutron deleted interface eea29fce-f6e3-4715-9a46-2a638462c6f8; detaching it from the instance and deleting it from the info cache
Apr 14 04:19:42 user nova-compute[71317]: DEBUG nova.network.neutron [req-410ee83a-1aaa-4a13-8cac-4a563317a793 req-4e73289a-bc89-4d56-b754-801eab0e7a77 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating instance_info_cache with network_info: [{"id": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "address": "fa:16:3e:b3:13:af", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.66", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c995dcb-b4", "ovs_interfaceid": "8c995dcb-b43b-4bdd-b631-e3b6104d49d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "address": "fa:16:3e:86:19:cc", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.209", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc9994009-b1", "ovs_interfaceid": "c9994009-b14a-4ff4-ac36-f02b0feccfb6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "address": "fa:16:3e:48:3e:f5", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.147", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap11fd6438-8a", "ovs_interfaceid": "11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "address": "fa:16:3e:95:73:7e", "network": {"id": "ca386160-6d9c-4824-a9c3-a50508890008", "bridge": "br-int", "label": "tempest-device-tagging-net1-507891129", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.153", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8257c16c-61", "ovs_interfaceid": "8257c16c-6115-4b67-a739-91cb1d5e8bc6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "address": "fa:16:3e:9c:4f:8e", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07a4c8ef-59", "ovs_interfaceid": "07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "address": "fa:16:3e:53:d2:74", "network": {"id": "a05db065-8e40-45f0-b7b3-49aa6fa45eed", "bridge": "br-int", "label": "tempest-device-tagging-net2-882249872", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "225e61ca983d4f27adc61ad33c81fce4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap512baf47-b5", "ovs_interfaceid": "512baf47-b5bf-4cf0-8300-f0e0167fc70e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:19:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-410ee83a-1aaa-4a13-8cac-4a563317a793 req-4e73289a-bc89-4d56-b754-801eab0e7a77 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Detach interface failed, port_id=eea29fce-f6e3-4715-9a46-2a638462c6f8, reason: Instance 55ba1cae-2b2f-43de-86db-996a8791fb02 could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:19:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:43 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:19:43 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Took 3.87 seconds to deallocate network for instance.
Apr 14 04:19:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:44 user nova-compute[71317]: DEBUG nova.compute.manager [req-03198ff2-c560-479b-a571-93b2aba8a6bc req-8dde7324-f62f-4e08-ad58-c8081f171895 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-deleted-11fd6438-8a9b-4c97-8a6f-da3ddfc36ebe {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:44 user nova-compute[71317]: DEBUG nova.compute.manager [req-03198ff2-c560-479b-a571-93b2aba8a6bc req-8dde7324-f62f-4e08-ad58-c8081f171895 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-deleted-07a4c8ef-59a4-4112-8dd7-0e5ffd9021a5 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:44 user nova-compute[71317]: DEBUG nova.compute.manager [req-03198ff2-c560-479b-a571-93b2aba8a6bc req-8dde7324-f62f-4e08-ad58-c8081f171895 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-deleted-8257c16c-6115-4b67-a739-91cb1d5e8bc6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:44 user nova-compute[71317]: DEBUG nova.compute.manager [req-03198ff2-c560-479b-a571-93b2aba8a6bc req-8dde7324-f62f-4e08-ad58-c8081f171895 service nova] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Received event network-vif-deleted-512baf47-b5bf-4cf0-8300-f0e0167fc70e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:19:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:46 user nova-compute[71317]: INFO nova.compute.manager [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Took 2.55 seconds to detach 3 volumes for instance.
Apr 14 04:19:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:46 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:19:47 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:19:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.462s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:47 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Deleted allocations for instance 55ba1cae-2b2f-43de-86db-996a8791fb02
Apr 14 04:19:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-dc9f897b-1670-4413-81a6-b50aae1af03b tempest-TaggedBootDevicesTest_v242-581315356 tempest-TaggedBootDevicesTest_v242-581315356-project-member] Lock "55ba1cae-2b2f-43de-86db-996a8791fb02" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 10.706s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:48 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'flavor' on Instance uuid 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.054s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:19:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:19:48 user nova-compute[71317]: INFO nova.compute.manager [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Attaching volume 70317b0d-2515-45ac-825b-207626822497 to /dev/vdb
Apr 14 04:19:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG os_brick.utils [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c3f0e7e-fe92-4631-919b-b55d4b0f9565]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[99202718-acec-4eea-a97a-290f6302824f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[117da7e5-47cb-4724-b773-88eae19c91b6]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1ce4932-a536-4f0c-a5ed-d6974c4fef0c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:19:49 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:19:49 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6572ea86-b561-4465-88cf-9da2ee14833c]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG os_brick.utils [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] <== get_connector_properties: return (124ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updating existing volume attachment record: ff3d8abd-b66f-43ca-baed-8c5b606bd22b {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:19:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681445978.464165, 55ba1cae-2b2f-43de-86db-996a8791fb02 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:19:53 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] VM Stopped (Lifecycle Event)
Apr 14 04:19:53 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bc175745-805d-410a-9ad2-9d3cfba57110 None None] [instance: 55ba1cae-2b2f-43de-86db-996a8791fb02] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HkOfdlri', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:19:53 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:53 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:19:53 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3155d37e-b7f6-4471-b9e1-b2739634e91b]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:19:53 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a306a97-c4c3-4f77-8a83-eeeba2dae5aa]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497] added
Apr 14 04:19:53 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[18d63e5e-d02e-4634-90c7-bc805dfec0a4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[44174851-8ce4-4daf-9647-b61d2732574b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HkOfdlri {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HkOfdlri" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b9ef7b9-8d3d-4d83-a577-5c37ce507b1a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'HkOfdlri'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4fb8a2bc-2308-4ade-bc9a-fafdb9ac8b87]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[39da375e-4621-48bb-953e-151ebcc462be]: (4, ('tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:53 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsi session list stdout=tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:53 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --login" returned: 0 in 0.068s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9f86d90-b2c4-4643-b325-e6c7b57bb38a]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260]
Apr 14 04:19:53 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260] successful.
Apr 14 04:19:53 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3404006-08fc-4dde-8cd4-81c6b9d86136]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b05820b-03ef-4dd8-aabc-2d668e82376f]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:19:53 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:53 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:19:53 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:19:53 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.285s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] HCTL ('33', '-', '-', 0) found on session 12 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[34ac5924-8b85-4328-a549-17d2e70de8c2]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Searching for a device in session 12 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HkOfdlri', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:19:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.317s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:19:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] <== connect_volume: return (1319ms) {'type': 'block', 'scsi_wwn': '23133643332343064', 'path': '/dev/sdb'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:19:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23133643332343064', 'path': '/dev/sdb'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:19:54 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'flavor' on Instance uuid 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:19:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:19:54 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:19:54 user nova-compute[71317]:   <source dev="/dev/sdb"/>
Apr 14 04:19:54 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:19:54 user nova-compute[71317]:   <serial>70317b0d-2515-45ac-825b-207626822497</serial>
Apr 14 04:19:54 user nova-compute[71317]: </disk>
Apr 14 04:19:54 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:19:55 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:19:55 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:19:55 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] No VIF found with MAC fa:16:3e:0f:a8:af, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:19:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ebc290f9-6648-4d11-bda1-1a8ae965d20c tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.467s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:19:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:19:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-51daa497-ae8f-43ab-8a4f-2ab88c08900f req-7993aecd-3f3f-467f-aa13-b97e9f34b606 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-changed-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-51daa497-ae8f-43ab-8a4f-2ab88c08900f req-7993aecd-3f3f-467f-aa13-b97e9f34b606 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Refreshing instance network info cache due to event network-changed-ef7670a9-3421-45de-a1cd-976ddb46f2a1. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:20:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-51daa497-ae8f-43ab-8a4f-2ab88c08900f req-7993aecd-3f3f-467f-aa13-b97e9f34b606 service nova] Acquiring lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-51daa497-ae8f-43ab-8a4f-2ab88c08900f req-7993aecd-3f3f-467f-aa13-b97e9f34b606 service nova] Acquired lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:01 user nova-compute[71317]: DEBUG nova.network.neutron [req-51daa497-ae8f-43ab-8a4f-2ab88c08900f req-7993aecd-3f3f-467f-aa13-b97e9f34b606 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Refreshing network info cache for port ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:20:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:01 user nova-compute[71317]: INFO nova.compute.manager [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Terminating instance
Apr 14 04:20:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:02 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Instance destroyed successfully.
Apr 14 04:20:02 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lazy-loading 'resources' on Instance uuid 8ed1dea8-e48a-49f7-924e-82cf639a1677 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-420804238',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-420804238',id=10,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBvOf9hss+Yqylm6PnX6la1ymm/55r9GRsHJ0NsNe4bgFVG/3wZy3kBc49f/xLbrJ51dzLLHr/o6IQVXcYhlDKHgp4DCKWnVv/anFtE+/Na/vdZgAYvQkLcZYHFyD4dFRQ==',key_name='tempest-keypair-330344245',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='01cefa66350a4432ad71204b39e953be',ramdisk_id='',reservation_id='r-p91nk3xp',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-926384577',owner_user_name='tempest-TaggedAttachmentsTest-926384577-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:17:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='0aeeeae7cbc340daa8cd81c1418794e0',uuid=8ed1dea8-e48a-49f7-924e-82cf639a1677,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.88", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converting VIF {"id": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "address": "fa:16:3e:58:93:64", "network": {"id": "c9babf6f-d879-4cff-b628-f100d9293bff", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1516755738-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.88", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01cefa66350a4432ad71204b39e953be", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap60579bc4-de", "ovs_interfaceid": "60579bc4-de13-48d7-b42d-2b9fc5597e45", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:58:93:64,bridge_name='br-int',has_traffic_filtering=True,id=60579bc4-de13-48d7-b42d-2b9fc5597e45,network=Network(c9babf6f-d879-4cff-b628-f100d9293bff),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap60579bc4-de') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG os_vif [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:58:93:64,bridge_name='br-int',has_traffic_filtering=True,id=60579bc4-de13-48d7-b42d-2b9fc5597e45,network=Network(c9babf6f-d879-4cff-b628-f100d9293bff),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap60579bc4-de') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap60579bc4-de, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:02 user nova-compute[71317]: INFO os_vif [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:58:93:64,bridge_name='br-int',has_traffic_filtering=True,id=60579bc4-de13-48d7-b42d-2b9fc5597e45,network=Network(c9babf6f-d879-4cff-b628-f100d9293bff),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap60579bc4-de')
Apr 14 04:20:02 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Deleting instance files /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677_del
Apr 14 04:20:02 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Deletion of /opt/stack/data/nova/instances/8ed1dea8-e48a-49f7-924e-82cf639a1677_del complete
Apr 14 04:20:02 user nova-compute[71317]: DEBUG nova.network.neutron [req-51daa497-ae8f-43ab-8a4f-2ab88c08900f req-7993aecd-3f3f-467f-aa13-b97e9f34b606 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updated VIF entry in instance network info cache for port ef7670a9-3421-45de-a1cd-976ddb46f2a1. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG nova.network.neutron [req-51daa497-ae8f-43ab-8a4f-2ab88c08900f req-7993aecd-3f3f-467f-aa13-b97e9f34b606 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updating instance_info_cache with network_info: [{"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-51daa497-ae8f-43ab-8a4f-2ab88c08900f req-7993aecd-3f3f-467f-aa13-b97e9f34b606 service nova] Releasing lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:02 user nova-compute[71317]: INFO nova.compute.manager [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Took 0.99 seconds to destroy the instance on the hypervisor.
Apr 14 04:20:02 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:20:02 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:20:03 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:03 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Took 0.98 seconds to deallocate network for instance.
Apr 14 04:20:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-vif-unplugged-60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] No waiting events found dispatching network-vif-unplugged-60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-vif-unplugged-60579bc4-de13-48d7-b42d-2b9fc5597e45 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-vif-plugged-60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] Acquiring lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] No waiting events found dispatching network-vif-plugged-60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:04 user nova-compute[71317]: WARNING nova.compute.manager [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received unexpected event network-vif-plugged-60579bc4-de13-48d7-b42d-2b9fc5597e45 for instance with vm_state active and task_state deleting.
Apr 14 04:20:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-2deedece-3fd8-46bc-95d4-f86b7582f8da req-af0a57c8-29f0-4a89-be6e-1753742e4fe2 service nova] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Received event network-vif-deleted-60579bc4-de13-48d7-b42d-2b9fc5597e45 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.419s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:04 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Deleted allocations for instance 8ed1dea8-e48a-49f7-924e-82cf639a1677
Apr 14 04:20:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d37981da-5589-4cb9-82b6-7d96d273bda4 tempest-TaggedAttachmentsTest-926384577 tempest-TaggedAttachmentsTest-926384577-project-member] Lock "8ed1dea8-e48a-49f7-924e-82cf639a1677" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.584s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-d19708c9-980c-4225-aa59-8e051ca21566 req-282e88a7-7b06-45d3-bd86-74dbb68bebd2 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-changed-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-d19708c9-980c-4225-aa59-8e051ca21566 req-282e88a7-7b06-45d3-bd86-74dbb68bebd2 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Refreshing instance network info cache due to event network-changed-ef7670a9-3421-45de-a1cd-976ddb46f2a1. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d19708c9-980c-4225-aa59-8e051ca21566 req-282e88a7-7b06-45d3-bd86-74dbb68bebd2 service nova] Acquiring lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d19708c9-980c-4225-aa59-8e051ca21566 req-282e88a7-7b06-45d3-bd86-74dbb68bebd2 service nova] Acquired lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:04 user nova-compute[71317]: DEBUG nova.network.neutron [req-d19708c9-980c-4225-aa59-8e051ca21566 req-282e88a7-7b06-45d3-bd86-74dbb68bebd2 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Refreshing network info cache for port ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:20:05 user nova-compute[71317]: DEBUG nova.network.neutron [req-d19708c9-980c-4225-aa59-8e051ca21566 req-282e88a7-7b06-45d3-bd86-74dbb68bebd2 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updated VIF entry in instance network info cache for port ef7670a9-3421-45de-a1cd-976ddb46f2a1. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:20:05 user nova-compute[71317]: DEBUG nova.network.neutron [req-d19708c9-980c-4225-aa59-8e051ca21566 req-282e88a7-7b06-45d3-bd86-74dbb68bebd2 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updating instance_info_cache with network_info: [{"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d19708c9-980c-4225-aa59-8e051ca21566 req-282e88a7-7b06-45d3-bd86-74dbb68bebd2 service nova] Releasing lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "59e547de-a74b-4ccd-90de-e7b61cba6964" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:06 user nova-compute[71317]: INFO nova.compute.manager [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Terminating instance
Apr 14 04:20:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-c1379301-daa2-4363-a814-e8f558b2e3c9 req-ba547b6f-d6da-4f77-8f79-c6cdab9fdd29 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Received event network-vif-unplugged-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c1379301-daa2-4363-a814-e8f558b2e3c9 req-ba547b6f-d6da-4f77-8f79-c6cdab9fdd29 service nova] Acquiring lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c1379301-daa2-4363-a814-e8f558b2e3c9 req-ba547b6f-d6da-4f77-8f79-c6cdab9fdd29 service nova] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c1379301-daa2-4363-a814-e8f558b2e3c9 req-ba547b6f-d6da-4f77-8f79-c6cdab9fdd29 service nova] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-c1379301-daa2-4363-a814-e8f558b2e3c9 req-ba547b6f-d6da-4f77-8f79-c6cdab9fdd29 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] No waiting events found dispatching network-vif-unplugged-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-c1379301-daa2-4363-a814-e8f558b2e3c9 req-ba547b6f-d6da-4f77-8f79-c6cdab9fdd29 service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Received event network-vif-unplugged-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:07 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Instance destroyed successfully.
Apr 14 04:20:07 user nova-compute[71317]: DEBUG nova.objects.instance [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lazy-loading 'resources' on Instance uuid 59e547de-a74b-4ccd-90de-e7b61cba6964 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:18:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1774642635',display_name='tempest-ServersNegativeTestJSON-server-1774642635',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1774642635',id=14,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:22Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bae0fb4af555422a847620627dffeae8',ramdisk_id='',reservation_id='r-46d3609f',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServersNegativeTestJSON-406789078',owner_user_name='tempest-ServersNegativeTestJSON-406789078-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:18:23Z,user_data=None,user_id='ea68fb139b004f748cf1bcfb8d675a2e',uuid=59e547de-a74b-4ccd-90de-e7b61cba6964,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "address": "fa:16:3e:95:29:3f", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde9378d0-c6", "ovs_interfaceid": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:20:07 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Converting VIF {"id": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "address": "fa:16:3e:95:29:3f", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde9378d0-c6", "ovs_interfaceid": "de9378d0-c63f-48c3-9a25-e7d82b8c0f99", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:07 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:95:29:3f,bridge_name='br-int',has_traffic_filtering=True,id=de9378d0-c63f-48c3-9a25-e7d82b8c0f99,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapde9378d0-c6') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:07 user nova-compute[71317]: DEBUG os_vif [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:95:29:3f,bridge_name='br-int',has_traffic_filtering=True,id=de9378d0-c63f-48c3-9a25-e7d82b8c0f99,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapde9378d0-c6') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:20:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapde9378d0-c6, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:07 user nova-compute[71317]: INFO os_vif [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:95:29:3f,bridge_name='br-int',has_traffic_filtering=True,id=de9378d0-c63f-48c3-9a25-e7d82b8c0f99,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapde9378d0-c6')
Apr 14 04:20:07 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Deleting instance files /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964_del
Apr 14 04:20:07 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Deletion of /opt/stack/data/nova/instances/59e547de-a74b-4ccd-90de-e7b61cba6964_del complete
Apr 14 04:20:07 user nova-compute[71317]: INFO nova.compute.manager [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Took 0.92 seconds to destroy the instance on the hypervisor.
Apr 14 04:20:07 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:20:07 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:20:07 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:20:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:08 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:08 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Took 1.06 seconds to deallocate network for instance.
Apr 14 04:20:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5c7aaea5-2cb3-4d34-9227-6f063e325491 req-eee2fd55-d004-4b1f-b4b2-110a4617321e service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Received event network-vif-deleted-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" acquired by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:08 user nova-compute[71317]: INFO nova.compute.manager [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Rebooting instance
Apr 14 04:20:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquired lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:08 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:20:08 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:20:08 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:20:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.340s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:09 user nova-compute[71317]: DEBUG nova.compute.manager [req-d124ca1d-ff89-4700-b646-e0ef41835101 req-8a59cf03-317e-47cc-9bb2-a758805dbc5b service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Received event network-vif-plugged-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d124ca1d-ff89-4700-b646-e0ef41835101 req-8a59cf03-317e-47cc-9bb2-a758805dbc5b service nova] Acquiring lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d124ca1d-ff89-4700-b646-e0ef41835101 req-8a59cf03-317e-47cc-9bb2-a758805dbc5b service nova] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d124ca1d-ff89-4700-b646-e0ef41835101 req-8a59cf03-317e-47cc-9bb2-a758805dbc5b service nova] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:09 user nova-compute[71317]: DEBUG nova.compute.manager [req-d124ca1d-ff89-4700-b646-e0ef41835101 req-8a59cf03-317e-47cc-9bb2-a758805dbc5b service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] No waiting events found dispatching network-vif-plugged-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:09 user nova-compute[71317]: WARNING nova.compute.manager [req-d124ca1d-ff89-4700-b646-e0ef41835101 req-8a59cf03-317e-47cc-9bb2-a758805dbc5b service nova] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Received unexpected event network-vif-plugged-de9378d0-c63f-48c3-9a25-e7d82b8c0f99 for instance with vm_state deleted and task_state None.
Apr 14 04:20:09 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Deleted allocations for instance 59e547de-a74b-4ccd-90de-e7b61cba6964
Apr 14 04:20:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-908dab50-c367-4b55-a64b-a33de7f941b7 tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "59e547de-a74b-4ccd-90de-e7b61cba6964" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.518s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:09 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updating instance_info_cache with network_info: [{"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Releasing lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:09 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:10 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Instance destroyed successfully.
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'resources' on Instance uuid 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:17:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-99049874',display_name='tempest-TestMinimumBasicScenario-server-99049874',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-99049874',id=13,image_ref='2921b7b4-17d1-4c56-9fbc-a30733db1472',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA++2BXIcEQ2v7bj136wrEst3TLhMf3Pwm5yFI2y5OTUcyHd3G9mX9nGbcm6RmV/qF9Miy1upxS4ewe90Jmk1PeD7f+WKuozsw6VgX059vi7WiBA0frzDuQmMNJN3b4XIg==',key_name='tempest-TestMinimumBasicScenario-1599913910',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:00Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0a60f60717b44f4ba904a949eabfe608',ramdisk_id='',reservation_id='r-n2gdtvof',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='2921b7b4-17d1-4c56-9fbc-a30733db1472',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1872250892',owner_user_name='tempest-TestMinimumBasicScenario-1872250892-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:20:09Z,user_data=None,user_id='982f167333954612a51911fc8d6c1ce9',uuid=4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converting VIF {"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_vif [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapef7670a9-34, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:10 user nova-compute[71317]: INFO os_vif [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34')
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HkOfdlri', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:10 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e73ea52-1276-4702-b6e4-671121e2ed91]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:20:10 user nova-compute[71317]: No targets found.
Apr 14 04:20:10 user nova-compute[71317]: iSNS:
Apr 14 04:20:10 user nova-compute[71317]: No targets found.
Apr 14 04:20:10 user nova-compute[71317]: STATIC:
Apr 14 04:20:10 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497
Apr 14 04:20:10 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:10 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:10 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:20:10 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:10 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:10 user nova-compute[71317]: FIRMWARE:
Apr 14 04:20:10 user nova-compute[71317]: No targets found.
Apr 14 04:20:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:20:10 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:20:10 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5967218-c1dd-4a9a-ba6d-2ddfc53a84c1]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[41430191-4e21-4584-9604-a576bb327d35]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:10 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:10 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d180b640>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497'): ({'sdb'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Removing single pathed devices sdb {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ec14f08-83e3-48cb-aeb7-78e39d3c7b37]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[073fd36c-7550-4873-8bf3-6f03a7c14857]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Flushing IO for device /dev/sdb {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb7fcd38-c7af-4870-816c-664881845711]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[63639ff3-32d2-4c6d-b045-80de27808df4]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[33c0847b-f5cb-4810-a50f-b7e9157feae0]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --logout" returned: 0 in 0.066s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[29c6614b-d6c5-4d06-8beb-7ba93f1492cc]: (4, ('Logging out of session [sid: 12, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260]\nLogout of [sid: 12, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 12, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260]
Apr 14 04:20:10 user nova-compute[71317]: Logout of [sid: 12, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260] successful.
Apr 14 04:20:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fca3a8e2-caa9-4ba1-b386-b22cf7978d26]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.283s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] <== disconnect_volume: return (284ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Start _get_guest_xml network_info=[{"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=2921b7b4-17d1-4c56-9fbc-a30733db1472,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '2921b7b4-17d1-4c56-9fbc-a30733db1472'}], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': None, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HkOfdlri', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, 'status': 'reserved', 'instance': '4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e', 'attached_at': '', 'detached_at': '', 'volume_id': '70317b0d-2515-45ac-825b-207626822497', 'serial': '70317b0d-2515-45ac-825b-207626822497'}, 'attachment_id': 'ff3d8abd-b66f-43ca-baed-8c5b606bd22b', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:20:10 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:20:10 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=2921b7b4-17d1-4c56-9fbc-a30733db1472,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'vcpu_model' on Instance uuid 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HkOfdlri', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:10 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:10 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:10 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-21aa1605-b0ca-49af-8f01-af03c3980b2f req-65d515d8-5226-4b76-bd6e-9bcc799b4371 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-vif-unplugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-21aa1605-b0ca-49af-8f01-af03c3980b2f req-65d515d8-5226-4b76-bd6e-9bcc799b4371 service nova] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-21aa1605-b0ca-49af-8f01-af03c3980b2f req-65d515d8-5226-4b76-bd6e-9bcc799b4371 service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-21aa1605-b0ca-49af-8f01-af03c3980b2f req-65d515d8-5226-4b76-bd6e-9bcc799b4371 service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-21aa1605-b0ca-49af-8f01-af03c3980b2f req-65d515d8-5226-4b76-bd6e-9bcc799b4371 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] No waiting events found dispatching network-vif-unplugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:10 user nova-compute[71317]: WARNING nova.compute.manager [req-21aa1605-b0ca-49af-8f01-af03c3980b2f req-65d515d8-5226-4b76-bd6e-9bcc799b4371 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received unexpected event network-vif-unplugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 for instance with vm_state active and task_state reboot_started_hard.
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-21aa1605-b0ca-49af-8f01-af03c3980b2f req-65d515d8-5226-4b76-bd6e-9bcc799b4371 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-21aa1605-b0ca-49af-8f01-af03c3980b2f req-65d515d8-5226-4b76-bd6e-9bcc799b4371 service nova] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-21aa1605-b0ca-49af-8f01-af03c3980b2f req-65d515d8-5226-4b76-bd6e-9bcc799b4371 service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-21aa1605-b0ca-49af-8f01-af03c3980b2f req-65d515d8-5226-4b76-bd6e-9bcc799b4371 service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-21aa1605-b0ca-49af-8f01-af03c3980b2f req-65d515d8-5226-4b76-bd6e-9bcc799b4371 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] No waiting events found dispatching network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:10 user nova-compute[71317]: WARNING nova.compute.manager [req-21aa1605-b0ca-49af-8f01-af03c3980b2f req-65d515d8-5226-4b76-bd6e-9bcc799b4371 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received unexpected event network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 for instance with vm_state active and task_state reboot_started_hard.
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260" returned: 21 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d9c7f2e-b3e2-4d2c-a450-dfda36d519d5]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:20:10 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9fbf1981-0a8f-4236-9fb4-ebd4aeb738e1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497] added
Apr 14 04:20:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5bd4023-5531-4e95-b1c6-9a9c3854697b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[937806af-bb21-4075-996d-0c2b55986b3f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HkOfdlri {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HkOfdlri" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[05738a7b-ae3d-4ff9-a852-6b47215a67ec]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'HkOfdlri'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1680ead1-7cef-4b19-9d0f-b93bf0fe208a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d10b669-94a5-494d-9ffc-87682b3d872b]: (4, ('tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsi session list stdout=tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --login" returned: 0 in 0.061s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5dc17fe1-b818-4026-bcea-d272cacdfdbb]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260]
Apr 14 04:20:10 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260] successful.
Apr 14 04:20:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[61987892-996f-4a41-9e07-3398e4e45706]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[209d7d89-5f09-4983-8715-9d4ac01f4c90]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:10 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:10 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.268s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] HCTL ('33', '-', '-', 0) found on session 13 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[97e1b2a6-2dd7-4148-ab0a-8b03995350f4]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Searching for a device in session 13 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:20:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HkOfdlri', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:20:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.305s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] <== connect_volume: return (1306ms) {'type': 'block', 'scsi_wwn': '23133643332343064', 'path': '/dev/sdb'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23133643332343064', 'path': '/dev/sdb'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:20:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:17:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-99049874',display_name='tempest-TestMinimumBasicScenario-server-99049874',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-99049874',id=13,image_ref='2921b7b4-17d1-4c56-9fbc-a30733db1472',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA++2BXIcEQ2v7bj136wrEst3TLhMf3Pwm5yFI2y5OTUcyHd3G9mX9nGbcm6RmV/qF9Miy1upxS4ewe90Jmk1PeD7f+WKuozsw6VgX059vi7WiBA0frzDuQmMNJN3b4XIg==',key_name='tempest-TestMinimumBasicScenario-1599913910',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:00Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0a60f60717b44f4ba904a949eabfe608',ramdisk_id='',reservation_id='r-n2gdtvof',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='2921b7b4-17d1-4c56-9fbc-a30733db1472',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1872250892',owner_user_name='tempest-TestMinimumBasicScenario-1872250892-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:20:09Z,user_data=None,user_id='982f167333954612a51911fc8d6c1ce9',uuid=4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:20:11 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converting VIF {"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:11 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'pci_devices' on Instance uuid 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:20:11 user nova-compute[71317]:   <uuid>4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e</uuid>
Apr 14 04:20:11 user nova-compute[71317]:   <name>instance-0000000d</name>
Apr 14 04:20:11 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:20:11 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:20:11 user nova-compute[71317]:   <metadata>
Apr 14 04:20:11 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:20:11 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:20:11 user nova-compute[71317]:       <nova:name>tempest-TestMinimumBasicScenario-server-99049874</nova:name>
Apr 14 04:20:11 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:20:10</nova:creationTime>
Apr 14 04:20:11 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:20:11 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:20:11 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:20:11 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:20:11 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:20:11 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:20:11 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:20:11 user nova-compute[71317]:       <nova:owner>
Apr 14 04:20:11 user nova-compute[71317]:         <nova:user uuid="982f167333954612a51911fc8d6c1ce9">tempest-TestMinimumBasicScenario-1872250892-project-member</nova:user>
Apr 14 04:20:11 user nova-compute[71317]:         <nova:project uuid="0a60f60717b44f4ba904a949eabfe608">tempest-TestMinimumBasicScenario-1872250892</nova:project>
Apr 14 04:20:11 user nova-compute[71317]:       </nova:owner>
Apr 14 04:20:11 user nova-compute[71317]:       <nova:root type="image" uuid="2921b7b4-17d1-4c56-9fbc-a30733db1472"/>
Apr 14 04:20:11 user nova-compute[71317]:       <nova:ports>
Apr 14 04:20:11 user nova-compute[71317]:         <nova:port uuid="ef7670a9-3421-45de-a1cd-976ddb46f2a1">
Apr 14 04:20:11 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Apr 14 04:20:11 user nova-compute[71317]:         </nova:port>
Apr 14 04:20:11 user nova-compute[71317]:       </nova:ports>
Apr 14 04:20:11 user nova-compute[71317]:     </nova:instance>
Apr 14 04:20:11 user nova-compute[71317]:   </metadata>
Apr 14 04:20:11 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:20:11 user nova-compute[71317]:     <system>
Apr 14 04:20:11 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:20:11 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:20:11 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:20:11 user nova-compute[71317]:       <entry name="serial">4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e</entry>
Apr 14 04:20:11 user nova-compute[71317]:       <entry name="uuid">4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e</entry>
Apr 14 04:20:11 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:20:11 user nova-compute[71317]:     </system>
Apr 14 04:20:11 user nova-compute[71317]:   </sysinfo>
Apr 14 04:20:11 user nova-compute[71317]:   <os>
Apr 14 04:20:11 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:20:11 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:20:11 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:20:11 user nova-compute[71317]:   </os>
Apr 14 04:20:11 user nova-compute[71317]:   <features>
Apr 14 04:20:11 user nova-compute[71317]:     <acpi/>
Apr 14 04:20:11 user nova-compute[71317]:     <apic/>
Apr 14 04:20:11 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:20:11 user nova-compute[71317]:   </features>
Apr 14 04:20:11 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:20:11 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:20:11 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:20:11 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:20:11 user nova-compute[71317]:   </clock>
Apr 14 04:20:11 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:20:11 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:20:11 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:20:11 user nova-compute[71317]:   </cpu>
Apr 14 04:20:11 user nova-compute[71317]:   <devices>
Apr 14 04:20:11 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:20:11 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:20:11 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk"/>
Apr 14 04:20:11 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:20:11 user nova-compute[71317]:     </disk>
Apr 14 04:20:11 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:20:11 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:20:11 user nova-compute[71317]:       <source dev="/dev/sdb"/>
Apr 14 04:20:11 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:20:11 user nova-compute[71317]:       <serial>70317b0d-2515-45ac-825b-207626822497</serial>
Apr 14 04:20:11 user nova-compute[71317]:     </disk>
Apr 14 04:20:11 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:20:11 user nova-compute[71317]:       <mac address="fa:16:3e:0f:a8:af"/>
Apr 14 04:20:11 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:11 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:20:11 user nova-compute[71317]:       <target dev="tapef7670a9-34"/>
Apr 14 04:20:11 user nova-compute[71317]:     </interface>
Apr 14 04:20:11 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:20:11 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/console.log" append="off"/>
Apr 14 04:20:11 user nova-compute[71317]:     </serial>
Apr 14 04:20:11 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:20:11 user nova-compute[71317]:     <video>
Apr 14 04:20:11 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:11 user nova-compute[71317]:     </video>
Apr 14 04:20:11 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:20:11 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:20:11 user nova-compute[71317]:     </rng>
Apr 14 04:20:11 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:20:11 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:20:11 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:20:11 user nova-compute[71317]:     </memballoon>
Apr 14 04:20:11 user nova-compute[71317]:   </devices>
Apr 14 04:20:11 user nova-compute[71317]: </domain>
Apr 14 04:20:11 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:20:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'trusted_certs' on Instance uuid 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9 --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Checking if we can resize image /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Cannot resize image /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'migration_context' on Instance uuid 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:17:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-99049874',display_name='tempest-TestMinimumBasicScenario-server-99049874',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-99049874',id=13,image_ref='2921b7b4-17d1-4c56-9fbc-a30733db1472',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA++2BXIcEQ2v7bj136wrEst3TLhMf3Pwm5yFI2y5OTUcyHd3G9mX9nGbcm6RmV/qF9Miy1upxS4ewe90Jmk1PeD7f+WKuozsw6VgX059vi7WiBA0frzDuQmMNJN3b4XIg==',key_name='tempest-TestMinimumBasicScenario-1599913910',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:00Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=1,progress=0,project_id='0a60f60717b44f4ba904a949eabfe608',ramdisk_id='',reservation_id='r-n2gdtvof',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='2921b7b4-17d1-4c56-9fbc-a30733db1472',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1872250892',owner_user_name='tempest-TestMinimumBasicScenario-1872250892-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:09Z,user_data=None,user_id='982f167333954612a51911fc8d6c1ce9',uuid=4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converting VIF {"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG os_vif [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Plugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapef7670a9-34, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapef7670a9-34, col_values=(('external_ids', {'iface-id': 'ef7670a9-3421-45de-a1cd-976ddb46f2a1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:0f:a8:af', 'vm-uuid': '4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:12 user nova-compute[71317]: INFO os_vif [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Successfully plugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34')
Apr 14 04:20:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-69db8ff5-362b-49ca-a9f2-187335826bb5 req-da68d0c0-7d2f-44f0-93ed-22e793ef7b4b service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-69db8ff5-362b-49ca-a9f2-187335826bb5 req-da68d0c0-7d2f-44f0-93ed-22e793ef7b4b service nova] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-69db8ff5-362b-49ca-a9f2-187335826bb5 req-da68d0c0-7d2f-44f0-93ed-22e793ef7b4b service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-69db8ff5-362b-49ca-a9f2-187335826bb5 req-da68d0c0-7d2f-44f0-93ed-22e793ef7b4b service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-69db8ff5-362b-49ca-a9f2-187335826bb5 req-da68d0c0-7d2f-44f0-93ed-22e793ef7b4b service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] No waiting events found dispatching network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:14 user nova-compute[71317]: WARNING nova.compute.manager [req-69db8ff5-362b-49ca-a9f2-187335826bb5 req-da68d0c0-7d2f-44f0-93ed-22e793ef7b4b service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received unexpected event network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 for instance with vm_state active and task_state reboot_started_hard.
Apr 14 04:20:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:20:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:20:15 user nova-compute[71317]: INFO nova.compute.claims [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Claim successful on node user
Apr 14 04:20:15 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:20:15 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:20:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.424s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:20:16 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:20:16 user nova-compute[71317]: INFO nova.virt.block_device [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Booting with volume f2d01790-bd9f-4eff-aa4d-2248429aeb94 at /dev/vda
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446016.2981029, 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:16 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] VM Resumed (Lifecycle Event)
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:20:16 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Instance rebooted successfully.
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: reboot_started_hard, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG os_brick.utils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.policy [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3537afcf5a3e47deb53acb34d00e0e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3162a049880f4b85afda331443cf52f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b185e00b-d118-4f66-949d-78f711f624c5]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea58aad2-8619-4ec0-8f90-6e790524b911 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" "released" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: held 7.647s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446016.2994206, 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:16 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] VM Started (Lifecycle Event)
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ddaed3c7-9377-4ef1-bb55-7d71a13a0ba1]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[50e01d83-784b-443d-946c-cffce89efaeb]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ed001b5-93c6-4776-a8fe-8f8a3d249046]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:20:16 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[67ea08f9-ec58-4154-9aac-f15895009365]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG os_brick.utils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] <== get_connector_properties: return (132ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating existing volume attachment record: 6510384b-b7d8-458c-8e4c-bc14aa6c7bd4 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.compute.manager [req-ee892612-aa60-4470-ac7d-265e43d0f80c req-75860c08-b20b-4562-a0ae-dec9b46fb12b service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ee892612-aa60-4470-ac7d-265e43d0f80c req-75860c08-b20b-4562-a0ae-dec9b46fb12b service nova] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ee892612-aa60-4470-ac7d-265e43d0f80c req-75860c08-b20b-4562-a0ae-dec9b46fb12b service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ee892612-aa60-4470-ac7d-265e43d0f80c req-75860c08-b20b-4562-a0ae-dec9b46fb12b service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:16 user nova-compute[71317]: DEBUG nova.compute.manager [req-ee892612-aa60-4470-ac7d-265e43d0f80c req-75860c08-b20b-4562-a0ae-dec9b46fb12b service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] No waiting events found dispatching network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:16 user nova-compute[71317]: WARNING nova.compute.manager [req-ee892612-aa60-4470-ac7d-265e43d0f80c req-75860c08-b20b-4562-a0ae-dec9b46fb12b service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received unexpected event network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 for instance with vm_state active and task_state None.
Apr 14 04:20:17 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Successfully created port: 42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:20:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:17 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446002.8116171, 8ed1dea8-e48a-49f7-924e-82cf639a1677 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:17 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] VM Stopped (Lifecycle Event)
Apr 14 04:20:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bc4a67d2-04d5-43ef-9ce1-b2b07ad1517e None None] [instance: 8ed1dea8-e48a-49f7-924e-82cf639a1677] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Successfully created port: 8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:20:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Successfully created port: 97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:20:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:20 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Successfully created port: 62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:20:21 user nova-compute[71317]: INFO nova.virt.block_device [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Booting with volume 40e8dfdd-c6e4-4f7f-a465-913fe79829ac at /dev/vdb
Apr 14 04:20:21 user nova-compute[71317]: DEBUG os_brick.utils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6b51220-361f-4339-9433-3f02db8d2244]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3dbe146-4923-4b58-b302-f4f42992265e]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0522de74-7b57-4685-90dd-f80fae9d0c85]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e86fe1b0-8276-4ce7-b739-ca39c7bce4c5]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:20:21 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:20:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e21908d-35a2-4663-a09b-5e777f2e6598]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG os_brick.utils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] <== get_connector_properties: return (133ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating existing volume attachment record: c67610e7-bec4-4bc4-966e-21da3a7669f1 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:20:21 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Successfully created port: 7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:20:22 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Successfully updated port: 42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:20:22 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446007.3771327, 59e547de-a74b-4ccd-90de-e7b61cba6964 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:22 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] VM Stopped (Lifecycle Event)
Apr 14 04:20:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-f81b0160-8e0b-4bfa-b03c-bbe180f17cc3 None None] [instance: 59e547de-a74b-4ccd-90de-e7b61cba6964] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-e9a9359c-d2d2-4023-b916-b8dee7466ba1 req-48d69f85-52a9-4056-8cb9-8ef19d1f645e service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-changed-42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-e9a9359c-d2d2-4023-b916-b8dee7466ba1 req-48d69f85-52a9-4056-8cb9-8ef19d1f645e service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing instance network info cache due to event network-changed-42a38c57-4357-4a88-91e3-d76bb38534db. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:20:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e9a9359c-d2d2-4023-b916-b8dee7466ba1 req-48d69f85-52a9-4056-8cb9-8ef19d1f645e service nova] Acquiring lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e9a9359c-d2d2-4023-b916-b8dee7466ba1 req-48d69f85-52a9-4056-8cb9-8ef19d1f645e service nova] Acquired lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:22 user nova-compute[71317]: DEBUG nova.network.neutron [req-e9a9359c-d2d2-4023-b916-b8dee7466ba1 req-48d69f85-52a9-4056-8cb9-8ef19d1f645e service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing network info cache for port 42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:20:22 user nova-compute[71317]: DEBUG nova.network.neutron [req-e9a9359c-d2d2-4023-b916-b8dee7466ba1 req-48d69f85-52a9-4056-8cb9-8ef19d1f645e service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:20:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:22 user nova-compute[71317]: DEBUG nova.network.neutron [req-e9a9359c-d2d2-4023-b916-b8dee7466ba1 req-48d69f85-52a9-4056-8cb9-8ef19d1f645e service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e9a9359c-d2d2-4023-b916-b8dee7466ba1 req-48d69f85-52a9-4056-8cb9-8ef19d1f645e service nova] Releasing lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Successfully updated port: 186b0001-a3cf-40fa-babf-6b7045fbdb26 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:20:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Successfully updated port: 70e5d70d-d9af-4853-9de8-2cbff2433c60 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG nova.compute.manager [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-changed-186b0001-a3cf-40fa-babf-6b7045fbdb26 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG nova.compute.manager [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing instance network info cache due to event network-changed-186b0001-a3cf-40fa-babf-6b7045fbdb26. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] Acquiring lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] Acquired lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG nova.network.neutron [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing network info cache for port 186b0001-a3cf-40fa-babf-6b7045fbdb26 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG nova.network.neutron [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Successfully updated port: 8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG nova.network.neutron [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] Releasing lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG nova.compute.manager [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-changed-70e5d70d-d9af-4853-9de8-2cbff2433c60 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG nova.compute.manager [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing instance network info cache due to event network-changed-70e5d70d-d9af-4853-9de8-2cbff2433c60. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] Acquiring lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] Acquired lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG nova.network.neutron [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing network info cache for port 70e5d70d-d9af-4853-9de8-2cbff2433c60 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:20:24 user nova-compute[71317]: DEBUG nova.network.neutron [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:20:25 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Successfully updated port: 97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:20:25 user nova-compute[71317]: DEBUG nova.network.neutron [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e73c29ea-bc1e-4071-9c9d-fa7b325e3cff req-934a90ec-d3e7-4009-ba22-d200fffc3177 service nova] Releasing lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:26 user nova-compute[71317]: INFO nova.virt.block_device [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Booting with volume 5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 at /dev/vdc
Apr 14 04:20:26 user nova-compute[71317]: DEBUG os_brick.utils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Successfully updated port: 62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[678557cb-a875-4fcb-b9db-2d99526a6287]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[793d499a-adac-401a-8455-42c2d911394c]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[27fd249d-106d-4e4f-9dae-8f037272ed71]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[309334d6-5be0-49db-a073-1263808f3c79]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:20:26 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e3fc482-8fe8-4b4f-800b-122040d93102]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG os_brick.utils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] <== get_connector_properties: return (151ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating existing volume attachment record: 4259a8d9-79de-44fa-afcc-18b419dfaf6c {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-changed-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing instance network info cache due to event network-changed-8bfa317e-3995-442d-b5ab-cb1199cb1fb8. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] Acquiring lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] Acquired lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG nova.network.neutron [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing network info cache for port 8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG nova.network.neutron [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:20:26 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.network.neutron [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] Releasing lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-changed-97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing instance network info cache due to event network-changed-97ba6e98-092c-472f-9f27-eab96fe7f9ae. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] Acquiring lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] Acquired lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.network.neutron [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing network info cache for port 97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.network.neutron [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquired lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'info_cache' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.network.neutron [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] Releasing lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-changed-62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing instance network info cache due to event network-changed-62770b99-cf54-4132-ad17-d1859ddf8443. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] Acquiring lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] Acquired lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.network.neutron [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing network info cache for port 62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Successfully updated port: 7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:20:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.network.neutron [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:28 user nova-compute[71317]: INFO nova.compute.manager [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Terminating instance
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.network.neutron [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-35c24a45-932e-4278-8d49-f62e8c260e8d req-ad40ff8e-73d0-4a1a-acdb-732fe011d4e2 service nova] Releasing lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquired lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Updating instance_info_cache with network_info: [{"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-15ed8115-a241-4f1f-9eac-785dedb64fdf req-04b84fd0-8ad3-4791-835e-95c747bb08bb service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-changed-7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-15ed8115-a241-4f1f-9eac-785dedb64fdf req-04b84fd0-8ad3-4791-835e-95c747bb08bb service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing instance network info cache due to event network-changed-7249969b-00f8-4c4d-8573-832c3e0eb8fe. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-15ed8115-a241-4f1f-9eac-785dedb64fdf req-04b84fd0-8ad3-4791-835e-95c747bb08bb service nova] Acquiring lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-463798be-c3cf-410a-9297-8d878a1f7e36 req-06f04fc2-a88f-4e61-8462-a9eaaf19cd11 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Received event network-vif-unplugged-3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-463798be-c3cf-410a-9297-8d878a1f7e36 req-06f04fc2-a88f-4e61-8462-a9eaaf19cd11 service nova] Acquiring lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-463798be-c3cf-410a-9297-8d878a1f7e36 req-06f04fc2-a88f-4e61-8462-a9eaaf19cd11 service nova] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-463798be-c3cf-410a-9297-8d878a1f7e36 req-06f04fc2-a88f-4e61-8462-a9eaaf19cd11 service nova] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-463798be-c3cf-410a-9297-8d878a1f7e36 req-06f04fc2-a88f-4e61-8462-a9eaaf19cd11 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] No waiting events found dispatching network-vif-unplugged-3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-463798be-c3cf-410a-9297-8d878a1f7e36 req-06f04fc2-a88f-4e61-8462-a9eaaf19cd11 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Received event network-vif-unplugged-3f178241-75e2-4e0a-8d92-e647d05ed85e for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:20:28 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Instance destroyed successfully.
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.objects.instance [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lazy-loading 'resources' on Instance uuid b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:17:11Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-1758802671',display_name='tempest-TestVolumeSwap-server-1758802671',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumeswap-server-1758802671',id=11,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFbgdbG+/8WAlmHZQQgAOqS/UONGNW8t82kEvX2AWKVyIPVWzo6LlVeaPM998LCX+uM50LF1wQnPotgqjtWl9nx8bNCNwqqSA4Fxg3g55HZulk97rrGLje8WCx8z8LeJ+Q==',key_name='tempest-keypair-1360289330',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ffadee5eeb7f4b42ae93e2e2578ef897',ramdisk_id='',reservation_id='r-85pcdwwm',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeSwap-926061730',owner_user_name='tempest-TestVolumeSwap-926061730-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:17:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='1da030e7c9e44f9b82f3286378a2ede6',uuid=b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "address": "fa:16:3e:cd:d0:ea", "network": {"id": "43bde039-802b-47d6-9194-3fcc8d89149f", "bridge": "br-int", "label": "tempest-TestVolumeSwap-57267579-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.78", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ffadee5eeb7f4b42ae93e2e2578ef897", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3f178241-75", "ovs_interfaceid": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Converting VIF {"id": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "address": "fa:16:3e:cd:d0:ea", "network": {"id": "43bde039-802b-47d6-9194-3fcc8d89149f", "bridge": "br-int", "label": "tempest-TestVolumeSwap-57267579-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.78", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ffadee5eeb7f4b42ae93e2e2578ef897", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3f178241-75", "ovs_interfaceid": "3f178241-75e2-4e0a-8d92-e647d05ed85e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:cd:d0:ea,bridge_name='br-int',has_traffic_filtering=True,id=3f178241-75e2-4e0a-8d92-e647d05ed85e,network=Network(43bde039-802b-47d6-9194-3fcc8d89149f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3f178241-75') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG os_vif [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:cd:d0:ea,bridge_name='br-int',has_traffic_filtering=True,id=3f178241-75e2-4e0a-8d92-e647d05ed85e,network=Network(43bde039-802b-47d6-9194-3fcc8d89149f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3f178241-75') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3f178241-75, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:28 user nova-compute[71317]: INFO os_vif [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:cd:d0:ea,bridge_name='br-int',has_traffic_filtering=True,id=3f178241-75e2-4e0a-8d92-e647d05ed85e,network=Network(43bde039-802b-47d6-9194-3fcc8d89149f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3f178241-75')
Apr 14 04:20:28 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Deleting instance files /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7_del
Apr 14 04:20:28 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Deletion of /opt/stack/data/nova/instances/b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7_del complete
Apr 14 04:20:29 user nova-compute[71317]: INFO nova.compute.manager [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Took 0.96 seconds to destroy the instance on the hypervisor.
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updating instance_info_cache with network_info: [{"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Releasing lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance destroyed successfully.
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'resources' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1781752187',display_name='tempest-AttachVolumeTestJSON-server-1781752187',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1781752187',id=3,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBY2UT8M9EXLRm2waCk4jzyD2YisXrBUU8nZo6LT6g0AlR0LbKytQJ1iJL1X5k+1hK+8YQxwsn1kgdkATbgdoU6lyDlwBLIIaqdqxbWOahpeOBX2u8LLHQwm2K/x2XVeAg==',key_name='tempest-keypair-1428467741',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:16:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='6a89255339f74e55ad7c92967558bb81',ramdisk_id='',reservation_id='r-snsinryu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-991906288',owner_user_name='tempest-AttachVolumeTestJSON-991906288-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:18:38Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7610b45003e34f3395268977612f6c94',uuid=da3b4651-aaa0-430d-85b8-c15de386d08f,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converting VIF {"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_vif [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapdf63af46-27, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:29 user nova-compute[71317]: INFO os_vif [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27')
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mfOroQgL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:29 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7676dd05-a77d-43eb-b8a9-f41f33af72f0]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:20:29 user nova-compute[71317]: No targets found.
Apr 14 04:20:29 user nova-compute[71317]: iSNS:
Apr 14 04:20:29 user nova-compute[71317]: No targets found.
Apr 14 04:20:29 user nova-compute[71317]: STATIC:
Apr 14 04:20:29 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497
Apr 14 04:20:29 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:29 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:29 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:20:29 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:29 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:29 user nova-compute[71317]: FIRMWARE:
Apr 14 04:20:29 user nova-compute[71317]: No targets found.
Apr 14 04:20:29 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:20:29 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:20:29 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.180s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[87d6af5b-0094-4afd-b605-545c78ab1cf7]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.objects.instance [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'flavor' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a30319f-2d43-41f0-ba07-886828b4607e]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:29 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:29 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:29 user nova-compute[71317]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:29 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8490768940>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4'): ({'sde'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Removing single pathed devices sde {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0251850-b9c1-43d3-834f-e83159939dd1]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.074s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[52797c26-630c-4786-803e-b408dd07c033]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Flushing IO for device /dev/sde {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.177s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f50ffe4-5192-4739-a4a6-66a0a41a7bc4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.120s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ee2fb43-d494-4676-8a93-7503476fd4a5]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[37ad22db-2948-412c-81aa-7af2a408d0f0]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:29 user nova-compute[71317]: INFO nova.compute.manager [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Attaching volume 3e0082ad-1ad0-4980-a066-6017f65fce9e to /dev/vdb
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --logout" returned: 0 in 0.077s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[55dd1895-1114-4065-9015-b7a960f75fa6]: (4, ('Logging out of session [sid: 5, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260]\nLogout of [sid: 5, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 5, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260]
Apr 14 04:20:29 user nova-compute[71317]: Logout of [sid: 5, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260] successful.
Apr 14 04:20:29 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.utils [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[83ba0f7d-50ff-4c36-9daa-ce3c85601238]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.595s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] <== disconnect_volume: return (597ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f6644d1-9ac3-4fbe-bc49-202b8c75daf6]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Start _get_guest_xml network_info=[{"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': None, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mfOroQgL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, 'status': 'reserved', 'instance': 'da3b4651-aaa0-430d-85b8-c15de386d08f', 'attached_at': '', 'detached_at': '', 'volume_id': 'b7e94002-3eeb-442d-b224-ef83dacfa5a4', 'serial': 'b7e94002-3eeb-442d-b224-ef83dacfa5a4'}, 'attachment_id': 'f57596f4-3157-4f5b-b9aa-52470f8dbf1c', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'numa_topology' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[63f296b6-f5b3-4a40-b663-9794f4a4f2a0]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:29 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:20:29 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'vcpu_model' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f6a713b-8aa6-4716-ba54-4a8acdf8dede]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2579a8f2-dad4-4eb7-9c1a-9382f46c9d9d]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mfOroQgL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:30 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:30 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:30 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:20:30 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fbb878e4-f740-4f2b-b02d-0d4b486eb514]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.utils [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== get_connector_properties: return (214ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating existing volume attachment record: 4372af92-6441-4463-9a7f-3e4c116e5a00 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260" returned: 21 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd69b6fb-7bc1-4486-b1c7-8999f632d496]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:20:30 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG nova.compute.manager [req-d551fc2d-0d05-4606-b8b4-060685db7efb req-58769db3-e4c5-408d-90d6-9b28f63901b8 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Received event network-vif-deleted-3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:30 user nova-compute[71317]: INFO nova.compute.manager [req-d551fc2d-0d05-4606-b8b4-060685db7efb req-58769db3-e4c5-408d-90d6-9b28f63901b8 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Neutron deleted interface 3f178241-75e2-4e0a-8d92-e647d05ed85e; detaching it from the instance and deleting it from the info cache
Apr 14 04:20:30 user nova-compute[71317]: DEBUG nova.network.neutron [req-d551fc2d-0d05-4606-b8b4-060685db7efb req-58769db3-e4c5-408d-90d6-9b28f63901b8 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[54ecbbba-586b-4f71-87ef-8089052edceb]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4] added
Apr 14 04:20:30 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG nova.compute.manager [req-d551fc2d-0d05-4606-b8b4-060685db7efb req-58769db3-e4c5-408d-90d6-9b28f63901b8 service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Detach interface failed, port_id=3f178241-75e2-4e0a-8d92-e647d05ed85e, reason: Instance b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:20:30 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Took 1.12 seconds to deallocate network for instance.
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[200710ed-f8fd-4a4c-8928-14209cb6ced4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fed0760e-541a-425f-b1cf-fc0885d6e26a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mfOroQgL {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mfOroQgL" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec5cdcd9-aac5-4a72-bf1f-c4bcdea1d193]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'mfOroQgL'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c018a3de-7836-4a7e-b3e0-eba659f1c323]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f7edbaf-f6b3-4edb-9b17-dd2f3f6bd141]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:30 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:30 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --login" returned: 0 in 0.068s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e95fa9b-1016-4975-9f7f-39500fdc24d3]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260]
Apr 14 04:20:30 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260] successful.
Apr 14 04:20:30 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6f9aa37-b05b-4955-bfb0-f9b673346b2d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1630fbc2-19ee-4ba2-9aa3-55d290bbc7dc]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:30 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:30 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:30 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:30 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.446s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] HCTL ('34', '-', '-', 0) found on session 14 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d3ce634-5c95-4e90-b597-ee971f19ebf4]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Searching for a device in session 14 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mfOroQgL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.586s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG nova.compute.manager [req-cfbafd6f-2478-4177-bd74-1f8fc6b49863 req-6f3f9d09-48b4-4f24-a96e-34dac76c284d service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Received event network-vif-plugged-3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cfbafd6f-2478-4177-bd74-1f8fc6b49863 req-6f3f9d09-48b4-4f24-a96e-34dac76c284d service nova] Acquiring lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cfbafd6f-2478-4177-bd74-1f8fc6b49863 req-6f3f9d09-48b4-4f24-a96e-34dac76c284d service nova] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cfbafd6f-2478-4177-bd74-1f8fc6b49863 req-6f3f9d09-48b4-4f24-a96e-34dac76c284d service nova] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:30 user nova-compute[71317]: DEBUG nova.compute.manager [req-cfbafd6f-2478-4177-bd74-1f8fc6b49863 req-6f3f9d09-48b4-4f24-a96e-34dac76c284d service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] No waiting events found dispatching network-vif-plugged-3f178241-75e2-4e0a-8d92-e647d05ed85e {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:30 user nova-compute[71317]: WARNING nova.compute.manager [req-cfbafd6f-2478-4177-bd74-1f8fc6b49863 req-6f3f9d09-48b4-4f24-a96e-34dac76c284d service nova] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Received unexpected event network-vif-plugged-3f178241-75e2-4e0a-8d92-e647d05ed85e for instance with vm_state deleted and task_state None.
Apr 14 04:20:30 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Deleted allocations for instance b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:30 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Error from libvirt while getting description of instance-00000003: [Error Code 42] Domain not found: no domain with matching uuid 'da3b4651-aaa0-430d-85b8-c15de386d08f' (instance-00000003): libvirt.libvirtError: Domain not found: no domain with matching uuid 'da3b4651-aaa0-430d-85b8-c15de386d08f' (instance-00000003)
Apr 14 04:20:30 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Error from libvirt while getting description of instance-0000000b: [Error Code 42] Domain not found: no domain with matching uuid 'b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7' (instance-0000000b): libvirt.libvirtError: Domain not found: no domain with matching uuid 'b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7' (instance-0000000b)
Apr 14 04:20:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-83d7a229-1428-4277-b7de-4a4327dd81c2 tempest-TestVolumeSwap-926061730 tempest-TestVolumeSwap-926061730-project-member] Lock "b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.998s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.503s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] <== connect_volume: return (1506ms) {'type': 'block', 'scsi_wwn': '23463366364646231', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23463366364646231', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1781752187',display_name='tempest-AttachVolumeTestJSON-server-1781752187',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1781752187',id=3,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBY2UT8M9EXLRm2waCk4jzyD2YisXrBUU8nZo6LT6g0AlR0LbKytQJ1iJL1X5k+1hK+8YQxwsn1kgdkATbgdoU6lyDlwBLIIaqdqxbWOahpeOBX2u8LLHQwm2K/x2XVeAg==',key_name='tempest-keypair-1428467741',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:16:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='6a89255339f74e55ad7c92967558bb81',ramdisk_id='',reservation_id='r-snsinryu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-991906288',owner_user_name='tempest-AttachVolumeTestJSON-991906288-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:18:38Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7610b45003e34f3395268977612f6c94',uuid=da3b4651-aaa0-430d-85b8-c15de386d08f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converting VIF {"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'pci_devices' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:20:31 user nova-compute[71317]:   <uuid>da3b4651-aaa0-430d-85b8-c15de386d08f</uuid>
Apr 14 04:20:31 user nova-compute[71317]:   <name>instance-00000003</name>
Apr 14 04:20:31 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:20:31 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:20:31 user nova-compute[71317]:   <metadata>
Apr 14 04:20:31 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:20:31 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:20:31 user nova-compute[71317]:       <nova:name>tempest-AttachVolumeTestJSON-server-1781752187</nova:name>
Apr 14 04:20:31 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:20:29</nova:creationTime>
Apr 14 04:20:31 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:20:31 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:20:31 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:20:31 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:20:31 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:20:31 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:20:31 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:20:31 user nova-compute[71317]:       <nova:owner>
Apr 14 04:20:31 user nova-compute[71317]:         <nova:user uuid="7610b45003e34f3395268977612f6c94">tempest-AttachVolumeTestJSON-991906288-project-member</nova:user>
Apr 14 04:20:31 user nova-compute[71317]:         <nova:project uuid="6a89255339f74e55ad7c92967558bb81">tempest-AttachVolumeTestJSON-991906288</nova:project>
Apr 14 04:20:31 user nova-compute[71317]:       </nova:owner>
Apr 14 04:20:31 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:20:31 user nova-compute[71317]:       <nova:ports>
Apr 14 04:20:31 user nova-compute[71317]:         <nova:port uuid="df63af46-27d1-4086-ae2c-39e8bc829bc7">
Apr 14 04:20:31 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 04:20:31 user nova-compute[71317]:         </nova:port>
Apr 14 04:20:31 user nova-compute[71317]:       </nova:ports>
Apr 14 04:20:31 user nova-compute[71317]:     </nova:instance>
Apr 14 04:20:31 user nova-compute[71317]:   </metadata>
Apr 14 04:20:31 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:20:31 user nova-compute[71317]:     <system>
Apr 14 04:20:31 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:20:31 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:20:31 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:20:31 user nova-compute[71317]:       <entry name="serial">da3b4651-aaa0-430d-85b8-c15de386d08f</entry>
Apr 14 04:20:31 user nova-compute[71317]:       <entry name="uuid">da3b4651-aaa0-430d-85b8-c15de386d08f</entry>
Apr 14 04:20:31 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:20:31 user nova-compute[71317]:     </system>
Apr 14 04:20:31 user nova-compute[71317]:   </sysinfo>
Apr 14 04:20:31 user nova-compute[71317]:   <os>
Apr 14 04:20:31 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:20:31 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:20:31 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:20:31 user nova-compute[71317]:   </os>
Apr 14 04:20:31 user nova-compute[71317]:   <features>
Apr 14 04:20:31 user nova-compute[71317]:     <acpi/>
Apr 14 04:20:31 user nova-compute[71317]:     <apic/>
Apr 14 04:20:31 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:20:31 user nova-compute[71317]:   </features>
Apr 14 04:20:31 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:20:31 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:20:31 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:20:31 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:20:31 user nova-compute[71317]:   </clock>
Apr 14 04:20:31 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:20:31 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:20:31 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:20:31 user nova-compute[71317]:   </cpu>
Apr 14 04:20:31 user nova-compute[71317]:   <devices>
Apr 14 04:20:31 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:20:31 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:20:31 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk"/>
Apr 14 04:20:31 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:20:31 user nova-compute[71317]:     </disk>
Apr 14 04:20:31 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:20:31 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:20:31 user nova-compute[71317]:       <source dev="/dev/sdc"/>
Apr 14 04:20:31 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:20:31 user nova-compute[71317]:       <serial>b7e94002-3eeb-442d-b224-ef83dacfa5a4</serial>
Apr 14 04:20:31 user nova-compute[71317]:     </disk>
Apr 14 04:20:31 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:20:31 user nova-compute[71317]:       <mac address="fa:16:3e:68:ca:2f"/>
Apr 14 04:20:31 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:31 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:20:31 user nova-compute[71317]:       <target dev="tapdf63af46-27"/>
Apr 14 04:20:31 user nova-compute[71317]:     </interface>
Apr 14 04:20:31 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:20:31 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/console.log" append="off"/>
Apr 14 04:20:31 user nova-compute[71317]:     </serial>
Apr 14 04:20:31 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:20:31 user nova-compute[71317]:     <video>
Apr 14 04:20:31 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:31 user nova-compute[71317]:     </video>
Apr 14 04:20:31 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:20:31 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:20:31 user nova-compute[71317]:     </rng>
Apr 14 04:20:31 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:20:31 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:20:31 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:20:31 user nova-compute[71317]:     </memballoon>
Apr 14 04:20:31 user nova-compute[71317]:   </devices>
Apr 14 04:20:31 user nova-compute[71317]: </domain>
Apr 14 04:20:31 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:31 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating instance_info_cache with network_info: [{"id": "42a38c57-4357-4a88-91e3-d76bb38534db", "address": "fa:16:3e:42:d6:fc", "network": {"id": "75bed293-1d30-421a-9656-15f0ead0650b", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1531253437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap42a38c57-43", "ovs_interfaceid": "42a38c57-4357-4a88-91e3-d76bb38534db", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "address": "fa:16:3e:ab:44:84", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap186b0001-a3", "ovs_interfaceid": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "address": "fa:16:3e:27:63:d9", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.57", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap70e5d70d-d9", "ovs_interfaceid": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "address": "fa:16:3e:7e:6a:92", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.37", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8bfa317e-39", "ovs_interfaceid": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "address": "fa:16:3e:f7:31:a4", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.117", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap97ba6e98-09", "ovs_interfaceid": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "62770b99-cf54-4132-ad17-d1859ddf8443", "address": "fa:16:3e:b2:3f:e0", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap62770b99-cf", "ovs_interfaceid": "62770b99-cf54-4132-ad17-d1859ddf8443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "address": "fa:16:3e:06:02:5f", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7249969b-00", "ovs_interfaceid": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Releasing lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Instance network_info: |[{"id": "42a38c57-4357-4a88-91e3-d76bb38534db", "address": "fa:16:3e:42:d6:fc", "network": {"id": "75bed293-1d30-421a-9656-15f0ead0650b", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1531253437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap42a38c57-43", "ovs_interfaceid": "42a38c57-4357-4a88-91e3-d76bb38534db", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "address": "fa:16:3e:ab:44:84", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap186b0001-a3", "ovs_interfaceid": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "address": "fa:16:3e:27:63:d9", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.57", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap70e5d70d-d9", "ovs_interfaceid": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "address": "fa:16:3e:7e:6a:92", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.37", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8bfa317e-39", "ovs_interfaceid": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "address": "fa:16:3e:f7:31:a4", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.117", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap97ba6e98-09", "ovs_interfaceid": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "62770b99-cf54-4132-ad17-d1859ddf8443", "address": "fa:16:3e:b2:3f:e0", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap62770b99-cf", "ovs_interfaceid": "62770b99-cf54-4132-ad17-d1859ddf8443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "address": "fa:16:3e:06:02:5f", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7249969b-00", "ovs_interfaceid": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.798s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-15ed8115-a241-4f1f-9eac-785dedb64fdf req-04b84fd0-8ad3-4791-835e-95c747bb08bb service nova] Acquired lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG nova.network.neutron [req-15ed8115-a241-4f1f-9eac-785dedb64fdf req-04b84fd0-8ad3-4791-835e-95c747bb08bb service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing network info cache for port 7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'trusted_certs' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:20:32 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:20:32 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:20:32 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7835MB free_disk=26.528038024902344GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.240s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:20:32 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Creating image(s)
Apr 14 04:20:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Ensure instance console log exists: /opt/stack/data/nova/instances/3d9cd123-3212-4a1d-b7c8-cf0441101048/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Start _get_guest_xml network_info=[{"id": "42a38c57-4357-4a88-91e3-d76bb38534db", "address": "fa:16:3e:42:d6:fc", "network": {"id": "75bed293-1d30-421a-9656-15f0ead0650b", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1531253437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap42a38c57-43", "ovs_interfaceid": "42a38c57-4357-4a88-91e3-d76bb38534db", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "address": "fa:16:3e:ab:44:84", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap186b0001-a3", "ovs_interfaceid": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "address": "fa:16:3e:27:63:d9", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.57", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap70e5d70d-d9", "ovs_interfaceid": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "address": "fa:16:3e:7e:6a:92", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.37", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8bfa317e-39", "ovs_interfaceid": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "address": "fa:16:3e:f7:31:a4", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.117", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap97ba6e98-09", "ovs_interfaceid": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "62770b99-cf54-4132-ad17-d1859ddf8443", "address": "fa:16:3e:b2:3f:e0", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap62770b99-cf", "ovs_interfaceid": "62770b99-cf54-4132-ad17-d1859ddf8443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "address": "fa:16:3e:06:02:5f", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7249969b-00", "ovs_interfaceid": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk', 'boot_index': '2'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk', 'boot_index': '3'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FdyQafHs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '3d9cd123-3212-4a1d-b7c8-cf0441101048', 'attached_at': '', 'detached_at': '', 'volume_id': 'f2d01790-bd9f-4eff-aa4d-2248429aeb94', 'serial': 'f2d01790-bd9f-4eff-aa4d-2248429aeb94'}, 'attachment_id': '6510384b-b7d8-458c-8e4c-bc14aa6c7bd4', 'volume_type': None}, {'disk_bus': 'virtio', 'boot_index': 1, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'aEOFgACW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '3d9cd123-3212-4a1d-b7c8-cf0441101048', 'attached_at': '', 'detached_at': '', 'volume_id': '40e8dfdd-c6e4-4f7f-a465-913fe79829ac', 'serial': '40e8dfdd-c6e4-4f7f-a465-913fe79829ac'}, 'attachment_id': 'c67610e7-bec4-4bc4-966e-21da3a7669f1', 'volume_type': None}, {'disk_bus': 'virtio', 'boot_index': 2, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vdc', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'WspLiGZN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '3d9cd123-3212-4a1d-b7c8-cf0441101048', 'attached_at': '', 'detached_at': '', 'volume_id': '5e5988ce-8d0a-49a0-acc8-84f7d3917bd3', 'serial': '5e5988ce-8d0a-49a0-acc8-84f7d3917bd3'}, 'attachment_id': '4259a8d9-79de-44fa-afcc-18b419dfaf6c', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Checking if we can resize image /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:20:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:32 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:20:33 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FdyQafHs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:33 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:33 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:33 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 9b532a44-20b7-4590-babe-684c49598a62 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance da3b4651-aaa0-430d-85b8-c15de386d08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 313835fa-6405-41ba-bc89-c4bfc119326a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3d9cd123-3212-4a1d-b7c8-cf0441101048 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260" returned: 21 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c7c5cda-df31-4516-bfa3-7af7efd82b3d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Cannot resize image /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'migration_context' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:20:33 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance d2e6cf32-080b-47b2-b4db-2b776ba85b38 has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1689}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1781752187',display_name='tempest-AttachVolumeTestJSON-server-1781752187',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1781752187',id=3,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBY2UT8M9EXLRm2waCk4jzyD2YisXrBUU8nZo6LT6g0AlR0LbKytQJ1iJL1X5k+1hK+8YQxwsn1kgdkATbgdoU6lyDlwBLIIaqdqxbWOahpeOBX2u8LLHQwm2K/x2XVeAg==',key_name='tempest-keypair-1428467741',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:16:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=4,progress=0,project_id='6a89255339f74e55ad7c92967558bb81',ramdisk_id='',reservation_id='r-snsinryu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-991906288',owner_user_name='tempest-AttachVolumeTestJSON-991906288-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:18:38Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7610b45003e34f3395268977612f6c94',uuid=da3b4651-aaa0-430d-85b8-c15de386d08f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converting VIF {"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_vif [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4516c174-9ae4-4078-a883-90e13a878310]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94] added
Apr 14 04:20:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapdf63af46-27, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapdf63af46-27, col_values=(('external_ids', {'iface-id': 'df63af46-27d1-4086-ae2c-39e8bc829bc7', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:68:ca:2f', 'vm-uuid': 'da3b4651-aaa0-430d-85b8-c15de386d08f'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[50ed7bc9-f248-43ef-8766-029a8018320f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:33 user nova-compute[71317]: INFO os_vif [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27')
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[88822175-dad8-4ffb-bd10-8882d77b8bc1]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FdyQafHs {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FdyQafHs" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f393b5e-206b-4931-ae62-3f94c00a5569]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'FdyQafHs'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f66b3d57-b44b-4236-9033-d982590165c1]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4c567fa-beac-4ce5-8174-aec90862e5d6]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:33 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:33 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --login" returned: 0 in 0.148s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d98c3a04-c896-4309-8de2-986208fb47b9]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94, portal: 172.16.0.220,3260]
Apr 14 04:20:33 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94, portal: 172.16.0.220,3260] successful.
Apr 14 04:20:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d316f744-0525-4403-be2e-78a97f801727]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2b983b6-bc98-4266-8283-9cd3f31537dd]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:33 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:33 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:33 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:33 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.492s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] HCTL ('35', '-', '-', 0) found on session 15 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.network.neutron [req-15ed8115-a241-4f1f-9eac-785dedb64fdf req-04b84fd0-8ad3-4791-835e-95c747bb08bb service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updated VIF entry in instance network info cache for port 7249969b-00f8-4c4d-8573-832c3e0eb8fe. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.network.neutron [req-15ed8115-a241-4f1f-9eac-785dedb64fdf req-04b84fd0-8ad3-4791-835e-95c747bb08bb service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating instance_info_cache with network_info: [{"id": "42a38c57-4357-4a88-91e3-d76bb38534db", "address": "fa:16:3e:42:d6:fc", "network": {"id": "75bed293-1d30-421a-9656-15f0ead0650b", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1531253437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap42a38c57-43", "ovs_interfaceid": "42a38c57-4357-4a88-91e3-d76bb38534db", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "address": "fa:16:3e:ab:44:84", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap186b0001-a3", "ovs_interfaceid": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "address": "fa:16:3e:27:63:d9", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.57", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap70e5d70d-d9", "ovs_interfaceid": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "address": "fa:16:3e:7e:6a:92", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.37", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8bfa317e-39", "ovs_interfaceid": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "address": "fa:16:3e:f7:31:a4", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.117", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap97ba6e98-09", "ovs_interfaceid": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "62770b99-cf54-4132-ad17-d1859ddf8443", "address": "fa:16:3e:b2:3f:e0", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap62770b99-cf", "ovs_interfaceid": "62770b99-cf54-4132-ad17-d1859ddf8443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "address": "fa:16:3e:06:02:5f", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7249969b-00", "ovs_interfaceid": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c3ca343-453d-44ab-bed7-30a24e2026d6]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Searching for a device in session 15 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FdyQafHs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-15ed8115-a241-4f1f-9eac-785dedb64fdf req-04b84fd0-8ad3-4791-835e-95c747bb08bb service nova] Releasing lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.904s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.784s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:20:33 user nova-compute[71317]: INFO nova.compute.claims [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Claim successful on node user
Apr 14 04:20:34 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.634s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:20:34 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:20:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG nova.policy [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd2561bbc0c6d447da333ffdbb9fdc5f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d389bb78d2e40b9b07d2d3ecdb7566b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:20:34 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Creating image(s)
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "/opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "/opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "/opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.545s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] <== connect_volume: return (1546ms) {'type': 'block', 'scsi_wwn': '23233326362613236', 'path': '/dev/sdd'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23233326362613236', 'path': '/dev/sdd'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'aEOFgACW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:34 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:34 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:34 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260" returned: 21 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e86e612c-9f38-4ad2-a719-17bde6dcaec3]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:20:34 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e4fe051-8a40-4767-b2e7-1ba14c0ff24b]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac] added
Apr 14 04:20:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a06dfbc3-2759-4b67-894a-4219b660e373]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.193s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.035s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d173d5a-792c-4442-b974-c0917781b396]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op update -n node.session.auth.username -v aEOFgACW {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.211s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op update -n node.session.auth.username -v aEOFgACW" returned: 0 in 0.358s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7f07d7c-c7fb-46e9-b4c3-734c09b540bd]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'aEOFgACW'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.123s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcde435e-d695-434c-9897-9eb532068fef]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[083abbdc-d3dc-44fc-95ba-785d1f3c3ae1]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:35 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:35 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:35 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:35 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:35 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:35 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk 1073741824" returned: 0 in 0.470s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.685s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --login" returned: 0 in 0.113s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2541d465-d01b-444f-ba33-eeab9704c845]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac, portal: 172.16.0.220,3260]
Apr 14 04:20:35 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac, portal: 172.16.0.220,3260] successful.
Apr 14 04:20:35 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-a21545e0-238f-4187-89d9-3190538910c0 req-6865e67e-4412-4046-8b51-53048efad471 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a21545e0-238f-4187-89d9-3190538910c0 req-6865e67e-4412-4046-8b51-53048efad471 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a21545e0-238f-4187-89d9-3190538910c0 req-6865e67e-4412-4046-8b51-53048efad471 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a21545e0-238f-4187-89d9-3190538910c0 req-6865e67e-4412-4046-8b51-53048efad471 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-a21545e0-238f-4187-89d9-3190538910c0 req-6865e67e-4412-4046-8b51-53048efad471 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:35 user nova-compute[71317]: WARNING nova.compute.manager [req-a21545e0-238f-4187-89d9-3190538910c0 req-6865e67e-4412-4046-8b51-53048efad471 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state stopped and task_state powering-on.
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[85e274a8-0fb8-44c3-b859-02dd071ce524]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7bf409e3-6a9d-4e80-9773-6adcbc9eb68d]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:35 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:35 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:35 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:35 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:35 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:35 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:35 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:35 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.927s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] HCTL ('36', '-', '-', 0) found on session 16 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.174s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Checking if we can resize image /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.165s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b60b982e-6aad-4a3e-bca7-71800c05ec01]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Searching for a device in session 16 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'aEOFgACW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Cannot resize image /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lazy-loading 'migration_context' on Instance uuid d2e6cf32-080b-47b2-b4db-2b776ba85b38 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Ensure instance console log exists: /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Successfully created port: 7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YjWqRKtu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:36 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'flavor' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.066s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.118s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] <== connect_volume: return (2119ms) {'type': 'block', 'scsi_wwn': '23730313961363466', 'path': '/dev/sde'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23730313961363466', 'path': '/dev/sde'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.372s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:36 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:36 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260" returned: 21 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f79f0d7b-0f6f-41b3-a3f3-cd383f076068]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:20:36 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b234bcb2-0924-498c-af99-e3f9df8d99d4]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e] added
Apr 14 04:20:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:36 user nova-compute[71317]: INFO nova.compute.manager [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Attaching volume 3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 to /dev/vdb
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ce21684-0ebe-4917-aadc-dc1e6f71bc4a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae89ff1e-6d37-4478-8d0d-46d42b9585d5]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YjWqRKtu {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YjWqRKtu" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4236c2be-20fe-4c23-b3e0-ad451c8059b4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YjWqRKtu'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4867ed32-e73b-4047-9d33-665341b110de]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG os_brick.utils [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae47d99b-09ca-40c0-a88b-323fb256bd9d]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a14aed42-b86e-4656-ba22-b90c8a671430]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:37 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:37 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[99647436-1b12-473c-b4e0-b9066dd15d16]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[432be305-b215-4216-851b-27bb0b88ede7]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[87995ff5-f3bc-473c-85c8-9958d5c25d21]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --login" returned: 0 in 0.085s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b3cc299d-2c14-4117-8944-c4412df5c952]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260]
Apr 14 04:20:37 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260] successful.
Apr 14 04:20:37 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:20:37 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8aa3be19-550f-4411-bf78-8f7cf4211414]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.utils [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] <== get_connector_properties: return (137ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updating existing volume attachment record: f6c08d20-8171-4b4a-a116-07bc4dc24f0c {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ba571bc-daee-4a65-a7ef-10b5b0ab39e9]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Successfully updated port: 7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "refresh_cache-d2e6cf32-080b-47b2-b4db-2b776ba85b38" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquired lock "refresh_cache-d2e6cf32-080b-47b2-b4db-2b776ba85b38" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f91b74d4-90ee-4882-ab23-cc1a24ab084d]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-7637a7a1-5fe1-4813-955b-8d2135169398 req-3fe52637-6605-4b74-8189-5efa725a9395 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Received event network-changed-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-7637a7a1-5fe1-4813-955b-8d2135169398 req-3fe52637-6605-4b74-8189-5efa725a9395 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Refreshing instance network info cache due to event network-changed-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7637a7a1-5fe1-4813-955b-8d2135169398 req-3fe52637-6605-4b74-8189-5efa725a9395 service nova] Acquiring lock "refresh_cache-d2e6cf32-080b-47b2-b4db-2b776ba85b38" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:37 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:37 user nova-compute[71317]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:37 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.372s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] HCTL ('37', '-', '-', 0) found on session 17 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cce96681-5c50-4197-a801-c7f23e847fd9]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Searching for a device in session 17 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YjWqRKtu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Updating instance_info_cache with network_info: [{"id": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "address": "fa:16:3e:ae:9a:66", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7c8689e1-9c", "ovs_interfaceid": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446037.6174452, da3b4651-aaa0-430d-85b8-c15de386d08f => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:37 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] VM Resumed (Lifecycle Event)
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-e0503345-62b6-4b49-82cd-f4b9418a9c96 req-9196b38b-7568-4dac-91af-14b372673635 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e0503345-62b6-4b49-82cd-f4b9418a9c96 req-9196b38b-7568-4dac-91af-14b372673635 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e0503345-62b6-4b49-82cd-f4b9418a9c96 req-9196b38b-7568-4dac-91af-14b372673635 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e0503345-62b6-4b49-82cd-f4b9418a9c96 req-9196b38b-7568-4dac-91af-14b372673635 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-e0503345-62b6-4b49-82cd-f4b9418a9c96 req-9196b38b-7568-4dac-91af-14b372673635 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:37 user nova-compute[71317]: WARNING nova.compute.manager [req-e0503345-62b6-4b49-82cd-f4b9418a9c96 req-9196b38b-7568-4dac-91af-14b372673635 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state stopped and task_state powering-on.
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Releasing lock "refresh_cache-d2e6cf32-080b-47b2-b4db-2b776ba85b38" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Instance network_info: |[{"id": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "address": "fa:16:3e:ae:9a:66", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7c8689e1-9c", "ovs_interfaceid": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7637a7a1-5fe1-4813-955b-8d2135169398 req-3fe52637-6605-4b74-8189-5efa725a9395 service nova] Acquired lock "refresh_cache-d2e6cf32-080b-47b2-b4db-2b776ba85b38" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.network.neutron [req-7637a7a1-5fe1-4813-955b-8d2135169398 req-3fe52637-6605-4b74-8189-5efa725a9395 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Refreshing network info cache for port 7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Start _get_guest_xml network_info=[{"id": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "address": "fa:16:3e:ae:9a:66", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7c8689e1-9c", "ovs_interfaceid": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:20:37 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance rebooted successfully.
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'WspLiGZN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:37 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ee8968e5-91ad-4d32-bc29-3630324a6118 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: stopped, current task_state: powering-on, current DB power_state: 4, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:20:37 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:20:37 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:20:31Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1275756548',display_name='tempest-VolumesAdminNegativeTest-server-1275756548',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1275756548',id=17,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCLbK9oP2daoZ7CXzxqtKBII/FQaajUVtxz7jBM0XYl4vW+UAcK//Iu4dDRXCklmwqWmqltHv3bFd4W3dO1IesZBh1iAMKvrzpnwuq9+FegSMWhSVQUKbRRtM51bR4jMkw==',key_name='tempest-keypair-1913633949',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3d389bb78d2e40b9b07d2d3ecdb7566b',ramdisk_id='',reservation_id='r-0cwwro3v',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-618964915',owner_user_name='tempest-VolumesAdminNegativeTest-618964915-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:35Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d2561bbc0c6d447da333ffdbb9fdc5f4',uuid=d2e6cf32-080b-47b2-b4db-2b776ba85b38,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "address": "fa:16:3e:ae:9a:66", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7c8689e1-9c", "ovs_interfaceid": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Converting VIF {"id": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "address": "fa:16:3e:ae:9a:66", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7c8689e1-9c", "ovs_interfaceid": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ae:9a:66,bridge_name='br-int',has_traffic_filtering=True,id=7c8689e1-9c0a-42b6-8cb3-7751f619ba1f,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7c8689e1-9c') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lazy-loading 'pci_devices' on Instance uuid d2e6cf32-080b-47b2-b4db-2b776ba85b38 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:37 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] During sync_power_state the instance has a pending task (powering-on). Skip.
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446037.617953, da3b4651-aaa0-430d-85b8-c15de386d08f => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:37 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] VM Started (Lifecycle Event)
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:20:37 user nova-compute[71317]:   <uuid>d2e6cf32-080b-47b2-b4db-2b776ba85b38</uuid>
Apr 14 04:20:37 user nova-compute[71317]:   <name>instance-00000011</name>
Apr 14 04:20:37 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:20:37 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:20:37 user nova-compute[71317]:   <metadata>
Apr 14 04:20:37 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:20:37 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:20:37 user nova-compute[71317]:       <nova:name>tempest-VolumesAdminNegativeTest-server-1275756548</nova:name>
Apr 14 04:20:37 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:20:37</nova:creationTime>
Apr 14 04:20:37 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:20:37 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:20:37 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:20:37 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:20:37 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:20:37 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:20:37 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:20:37 user nova-compute[71317]:       <nova:owner>
Apr 14 04:20:37 user nova-compute[71317]:         <nova:user uuid="d2561bbc0c6d447da333ffdbb9fdc5f4">tempest-VolumesAdminNegativeTest-618964915-project-member</nova:user>
Apr 14 04:20:37 user nova-compute[71317]:         <nova:project uuid="3d389bb78d2e40b9b07d2d3ecdb7566b">tempest-VolumesAdminNegativeTest-618964915</nova:project>
Apr 14 04:20:37 user nova-compute[71317]:       </nova:owner>
Apr 14 04:20:37 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:20:37 user nova-compute[71317]:       <nova:ports>
Apr 14 04:20:37 user nova-compute[71317]:         <nova:port uuid="7c8689e1-9c0a-42b6-8cb3-7751f619ba1f">
Apr 14 04:20:37 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 04:20:37 user nova-compute[71317]:         </nova:port>
Apr 14 04:20:37 user nova-compute[71317]:       </nova:ports>
Apr 14 04:20:37 user nova-compute[71317]:     </nova:instance>
Apr 14 04:20:37 user nova-compute[71317]:   </metadata>
Apr 14 04:20:37 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:20:37 user nova-compute[71317]:     <system>
Apr 14 04:20:37 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:20:37 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:20:37 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:20:37 user nova-compute[71317]:       <entry name="serial">d2e6cf32-080b-47b2-b4db-2b776ba85b38</entry>
Apr 14 04:20:37 user nova-compute[71317]:       <entry name="uuid">d2e6cf32-080b-47b2-b4db-2b776ba85b38</entry>
Apr 14 04:20:37 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:20:37 user nova-compute[71317]:     </system>
Apr 14 04:20:37 user nova-compute[71317]:   </sysinfo>
Apr 14 04:20:37 user nova-compute[71317]:   <os>
Apr 14 04:20:37 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:20:37 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:20:37 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:20:37 user nova-compute[71317]:   </os>
Apr 14 04:20:37 user nova-compute[71317]:   <features>
Apr 14 04:20:37 user nova-compute[71317]:     <acpi/>
Apr 14 04:20:37 user nova-compute[71317]:     <apic/>
Apr 14 04:20:37 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:20:37 user nova-compute[71317]:   </features>
Apr 14 04:20:37 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:20:37 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:20:37 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:20:37 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:20:37 user nova-compute[71317]:   </clock>
Apr 14 04:20:37 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:20:37 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:20:37 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:20:37 user nova-compute[71317]:   </cpu>
Apr 14 04:20:37 user nova-compute[71317]:   <devices>
Apr 14 04:20:37 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:20:37 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:20:37 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk"/>
Apr 14 04:20:37 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:20:37 user nova-compute[71317]:     </disk>
Apr 14 04:20:37 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:20:37 user nova-compute[71317]:       <mac address="fa:16:3e:ae:9a:66"/>
Apr 14 04:20:37 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:37 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:20:37 user nova-compute[71317]:       <target dev="tap7c8689e1-9c"/>
Apr 14 04:20:37 user nova-compute[71317]:     </interface>
Apr 14 04:20:37 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:20:37 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/console.log" append="off"/>
Apr 14 04:20:37 user nova-compute[71317]:     </serial>
Apr 14 04:20:37 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:20:37 user nova-compute[71317]:     <video>
Apr 14 04:20:37 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:37 user nova-compute[71317]:     </video>
Apr 14 04:20:37 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:20:37 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:20:37 user nova-compute[71317]:     </rng>
Apr 14 04:20:37 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:20:37 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:20:37 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:20:37 user nova-compute[71317]:     </memballoon>
Apr 14 04:20:37 user nova-compute[71317]:   </devices>
Apr 14 04:20:37 user nova-compute[71317]: </domain>
Apr 14 04:20:37 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:20:31Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1275756548',display_name='tempest-VolumesAdminNegativeTest-server-1275756548',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1275756548',id=17,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCLbK9oP2daoZ7CXzxqtKBII/FQaajUVtxz7jBM0XYl4vW+UAcK//Iu4dDRXCklmwqWmqltHv3bFd4W3dO1IesZBh1iAMKvrzpnwuq9+FegSMWhSVQUKbRRtM51bR4jMkw==',key_name='tempest-keypair-1913633949',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3d389bb78d2e40b9b07d2d3ecdb7566b',ramdisk_id='',reservation_id='r-0cwwro3v',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-618964915',owner_user_name='tempest-VolumesAdminNegativeTest-618964915-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:35Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d2561bbc0c6d447da333ffdbb9fdc5f4',uuid=d2e6cf32-080b-47b2-b4db-2b776ba85b38,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "address": "fa:16:3e:ae:9a:66", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7c8689e1-9c", "ovs_interfaceid": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Converting VIF {"id": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "address": "fa:16:3e:ae:9a:66", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7c8689e1-9c", "ovs_interfaceid": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ae:9a:66,bridge_name='br-int',has_traffic_filtering=True,id=7c8689e1-9c0a-42b6-8cb3-7751f619ba1f,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7c8689e1-9c') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG os_vif [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ae:9a:66,bridge_name='br-int',has_traffic_filtering=True,id=7c8689e1-9c0a-42b6-8cb3-7751f619ba1f,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7c8689e1-9c') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7c8689e1-9c, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap7c8689e1-9c, col_values=(('external_ids', {'iface-id': '7c8689e1-9c0a-42b6-8cb3-7751f619ba1f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ae:9a:66', 'vm-uuid': 'd2e6cf32-080b-47b2-b4db-2b776ba85b38'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:37 user nova-compute[71317]: INFO os_vif [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ae:9a:66,bridge_name='br-int',has_traffic_filtering=True,id=7c8689e1-9c0a-42b6-8cb3-7751f619ba1f,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7c8689e1-9c')
Apr 14 04:20:37 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:20:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] No VIF found with MAC fa:16:3e:ae:9a:66, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.411s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== connect_volume: return (1783ms) {'type': 'block', 'scsi_wwn': '23362346236663531', 'path': '/dev/sdf'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23362346236663531', 'path': '/dev/sdf'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.576s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:38 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:38 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG nova.objects.instance [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'flavor' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG nova.network.neutron [req-7637a7a1-5fe1-4813-955b-8d2135169398 req-3fe52637-6605-4b74-8189-5efa725a9395 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Updated VIF entry in instance network info cache for port 7c8689e1-9c0a-42b6-8cb3-7751f619ba1f. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG nova.network.neutron [req-7637a7a1-5fe1-4813-955b-8d2135169398 req-3fe52637-6605-4b74-8189-5efa725a9395 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Updating instance_info_cache with network_info: [{"id": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "address": "fa:16:3e:ae:9a:66", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7c8689e1-9c", "ovs_interfaceid": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260" returned: 21 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[379a3dda-7a0c-49f6-9e3b-50d6060bfd16]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:20:38 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7637a7a1-5fe1-4813-955b-8d2135169398 req-3fe52637-6605-4b74-8189-5efa725a9395 service nova] Releasing lock "refresh_cache-d2e6cf32-080b-47b2-b4db-2b776ba85b38" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:20:38 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:20:38 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:20:38 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:20:38 user nova-compute[71317]:   <serial>3e0082ad-1ad0-4980-a066-6017f65fce9e</serial>
Apr 14 04:20:38 user nova-compute[71317]: </disk>
Apr 14 04:20:38 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f58a375a-82e0-4fb0-86dc-f360fe43d22d]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3] added
Apr 14 04:20:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[81363e3f-c4b7-417f-b443-5bc8f1b36576]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8c32c08-21e1-4db6-9a77-c4109ed6405e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v WspLiGZN {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v WspLiGZN" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1162f1a4-d902-41de-8258-94ed227138b9]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'WspLiGZN'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d399c1ff-1ac3-4043-bc35-06239a39ba83]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d1d15d7-efe0-4c42-bbd2-fe84c31ad3c4]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --login" returned: 0 in 0.094s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2e5fb89-e2e3-41cf-89db-d60688fdec25]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3, portal: 172.16.0.220,3260]
Apr 14 04:20:38 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3, portal: 172.16.0.220,3260] successful.
Apr 14 04:20:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[11ca1a1d-5067-476e-a8c2-937b86b1a1ff]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2ccd02f-34f6-4642-8683-8ecdcf16964a]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:38 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.361s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] HCTL ('38', '-', '-', 0) found on session 18 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae3c33b9-fed9-43b9-9cfb-1e4de55f1a5a]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Searching for a device in session 18 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:20:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Connected to sdg using {'target_iqn': 'iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'WspLiGZN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No VIF found with MAC fa:16:3e:5d:8f:83, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-68d3d448-ac20-4492-90c6-0067a190395f tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.808s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.435s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] <== connect_volume: return (2011ms) {'type': 'block', 'scsi_wwn': '26235613263376464', 'path': '/dev/sdg'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26235613263376464', 'path': '/dev/sdg'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "42a38c57-4357-4a88-91e3-d76bb38534db", "address": "fa:16:3e:42:d6:fc", "network": {"id": "75bed293-1d30-421a-9656-15f0ead0650b", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1531253437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap42a38c57-43", "ovs_interfaceid": "42a38c57-4357-4a88-91e3-d76bb38534db", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "42a38c57-4357-4a88-91e3-d76bb38534db", "address": "fa:16:3e:42:d6:fc", "network": {"id": "75bed293-1d30-421a-9656-15f0ead0650b", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1531253437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap42a38c57-43", "ovs_interfaceid": "42a38c57-4357-4a88-91e3-d76bb38534db", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:42:d6:fc,bridge_name='br-int',has_traffic_filtering=True,id=42a38c57-4357-4a88-91e3-d76bb38534db,network=Network(75bed293-1d30-421a-9656-15f0ead0650b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap42a38c57-43') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "address": "fa:16:3e:ab:44:84", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap186b0001-a3", "ovs_interfaceid": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "address": "fa:16:3e:ab:44:84", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap186b0001-a3", "ovs_interfaceid": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ab:44:84,bridge_name='br-int',has_traffic_filtering=True,id=186b0001-a3cf-40fa-babf-6b7045fbdb26,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap186b0001-a3') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "address": "fa:16:3e:27:63:d9", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.57", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap70e5d70d-d9", "ovs_interfaceid": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "address": "fa:16:3e:27:63:d9", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.57", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap70e5d70d-d9", "ovs_interfaceid": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:27:63:d9,bridge_name='br-int',has_traffic_filtering=True,id=70e5d70d-d9af-4853-9de8-2cbff2433c60,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap70e5d70d-d9') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "address": "fa:16:3e:7e:6a:92", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.37", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8bfa317e-39", "ovs_interfaceid": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "address": "fa:16:3e:7e:6a:92", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.37", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8bfa317e-39", "ovs_interfaceid": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7e:6a:92,bridge_name='br-int',has_traffic_filtering=True,id=8bfa317e-3995-442d-b5ab-cb1199cb1fb8,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8bfa317e-39') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "address": "fa:16:3e:f7:31:a4", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.117", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap97ba6e98-09", "ovs_interfaceid": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "address": "fa:16:3e:f7:31:a4", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.117", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap97ba6e98-09", "ovs_interfaceid": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=97ba6e98-092c-472f-9f27-eab96fe7f9ae,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap97ba6e98-09') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "62770b99-cf54-4132-ad17-d1859ddf8443", "address": "fa:16:3e:b2:3f:e0", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap62770b99-cf", "ovs_interfaceid": "62770b99-cf54-4132-ad17-d1859ddf8443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "62770b99-cf54-4132-ad17-d1859ddf8443", "address": "fa:16:3e:b2:3f:e0", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap62770b99-cf", "ovs_interfaceid": "62770b99-cf54-4132-ad17-d1859ddf8443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b2:3f:e0,bridge_name='br-int',has_traffic_filtering=True,id=62770b99-cf54-4132-ad17-d1859ddf8443,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap62770b99-cf') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "address": "fa:16:3e:06:02:5f", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7249969b-00", "ovs_interfaceid": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "address": "fa:16:3e:06:02:5f", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7249969b-00", "ovs_interfaceid": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:06:02:5f,bridge_name='br-int',has_traffic_filtering=True,id=7249969b-00f8-4c4d-8573-832c3e0eb8fe,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7249969b-00') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.objects.instance [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lazy-loading 'pci_devices' on Instance uuid 3d9cd123-3212-4a1d-b7c8-cf0441101048 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:20:39 user nova-compute[71317]:   <uuid>3d9cd123-3212-4a1d-b7c8-cf0441101048</uuid>
Apr 14 04:20:39 user nova-compute[71317]:   <name>instance-00000010</name>
Apr 14 04:20:39 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:20:39 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:20:39 user nova-compute[71317]:   <metadata>
Apr 14 04:20:39 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:20:39 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:20:39 user nova-compute[71317]:       <nova:name>tempest-device-tagging-server-313960955</nova:name>
Apr 14 04:20:39 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:20:33</nova:creationTime>
Apr 14 04:20:39 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:20:39 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:20:39 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:20:39 user nova-compute[71317]:       <nova:owner>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:user uuid="3537afcf5a3e47deb53acb34d00e0e83">tempest-TaggedBootDevicesTest-914882220-project-member</nova:user>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:project uuid="3162a049880f4b85afda331443cf52f1">tempest-TaggedBootDevicesTest-914882220</nova:project>
Apr 14 04:20:39 user nova-compute[71317]:       </nova:owner>
Apr 14 04:20:39 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:20:39 user nova-compute[71317]:       <nova:ports>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:port uuid="42a38c57-4357-4a88-91e3-d76bb38534db">
Apr 14 04:20:39 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 14 04:20:39 user nova-compute[71317]:         </nova:port>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:port uuid="186b0001-a3cf-40fa-babf-6b7045fbdb26">
Apr 14 04:20:39 user nova-compute[71317]:           <nova:ip type="fixed" address="10.1.1.100" ipVersion="4"/>
Apr 14 04:20:39 user nova-compute[71317]:         </nova:port>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:port uuid="70e5d70d-d9af-4853-9de8-2cbff2433c60">
Apr 14 04:20:39 user nova-compute[71317]:           <nova:ip type="fixed" address="10.1.1.57" ipVersion="4"/>
Apr 14 04:20:39 user nova-compute[71317]:         </nova:port>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:port uuid="8bfa317e-3995-442d-b5ab-cb1199cb1fb8">
Apr 14 04:20:39 user nova-compute[71317]:           <nova:ip type="fixed" address="10.1.1.37" ipVersion="4"/>
Apr 14 04:20:39 user nova-compute[71317]:         </nova:port>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:port uuid="97ba6e98-092c-472f-9f27-eab96fe7f9ae">
Apr 14 04:20:39 user nova-compute[71317]:           <nova:ip type="fixed" address="10.1.1.117" ipVersion="4"/>
Apr 14 04:20:39 user nova-compute[71317]:         </nova:port>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:port uuid="62770b99-cf54-4132-ad17-d1859ddf8443">
Apr 14 04:20:39 user nova-compute[71317]:           <nova:ip type="fixed" address="10.2.2.100" ipVersion="4"/>
Apr 14 04:20:39 user nova-compute[71317]:         </nova:port>
Apr 14 04:20:39 user nova-compute[71317]:         <nova:port uuid="7249969b-00f8-4c4d-8573-832c3e0eb8fe">
Apr 14 04:20:39 user nova-compute[71317]:           <nova:ip type="fixed" address="10.2.2.200" ipVersion="4"/>
Apr 14 04:20:39 user nova-compute[71317]:         </nova:port>
Apr 14 04:20:39 user nova-compute[71317]:       </nova:ports>
Apr 14 04:20:39 user nova-compute[71317]:     </nova:instance>
Apr 14 04:20:39 user nova-compute[71317]:   </metadata>
Apr 14 04:20:39 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:20:39 user nova-compute[71317]:     <system>
Apr 14 04:20:39 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:20:39 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:20:39 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:20:39 user nova-compute[71317]:       <entry name="serial">3d9cd123-3212-4a1d-b7c8-cf0441101048</entry>
Apr 14 04:20:39 user nova-compute[71317]:       <entry name="uuid">3d9cd123-3212-4a1d-b7c8-cf0441101048</entry>
Apr 14 04:20:39 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:20:39 user nova-compute[71317]:     </system>
Apr 14 04:20:39 user nova-compute[71317]:   </sysinfo>
Apr 14 04:20:39 user nova-compute[71317]:   <os>
Apr 14 04:20:39 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:20:39 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:20:39 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:20:39 user nova-compute[71317]:   </os>
Apr 14 04:20:39 user nova-compute[71317]:   <features>
Apr 14 04:20:39 user nova-compute[71317]:     <acpi/>
Apr 14 04:20:39 user nova-compute[71317]:     <apic/>
Apr 14 04:20:39 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:20:39 user nova-compute[71317]:   </features>
Apr 14 04:20:39 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:20:39 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:20:39 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:20:39 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:20:39 user nova-compute[71317]:   </clock>
Apr 14 04:20:39 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:20:39 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:20:39 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:20:39 user nova-compute[71317]:   </cpu>
Apr 14 04:20:39 user nova-compute[71317]:   <devices>
Apr 14 04:20:39 user nova-compute[71317]:     <disk type="file" device="cdrom">
Apr 14 04:20:39 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none"/>
Apr 14 04:20:39 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/3d9cd123-3212-4a1d-b7c8-cf0441101048/disk.config"/>
Apr 14 04:20:39 user nova-compute[71317]:       <target dev="hda" bus="ide"/>
Apr 14 04:20:39 user nova-compute[71317]:     </disk>
Apr 14 04:20:39 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:20:39 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:20:39 user nova-compute[71317]:       <source dev="/dev/sdd"/>
Apr 14 04:20:39 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:20:39 user nova-compute[71317]:       <serial>f2d01790-bd9f-4eff-aa4d-2248429aeb94</serial>
Apr 14 04:20:39 user nova-compute[71317]:     </disk>
Apr 14 04:20:39 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:20:39 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:20:39 user nova-compute[71317]:       <source dev="/dev/sde"/>
Apr 14 04:20:39 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:20:39 user nova-compute[71317]:       <serial>40e8dfdd-c6e4-4f7f-a465-913fe79829ac</serial>
Apr 14 04:20:39 user nova-compute[71317]:     </disk>
Apr 14 04:20:39 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:20:39 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:20:39 user nova-compute[71317]:       <source dev="/dev/sdg"/>
Apr 14 04:20:39 user nova-compute[71317]:       <target dev="vdc" bus="virtio"/>
Apr 14 04:20:39 user nova-compute[71317]:       <serial>5e5988ce-8d0a-49a0-acc8-84f7d3917bd3</serial>
Apr 14 04:20:39 user nova-compute[71317]:     </disk>
Apr 14 04:20:39 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:20:39 user nova-compute[71317]:       <mac address="fa:16:3e:42:d6:fc"/>
Apr 14 04:20:39 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:39 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:20:39 user nova-compute[71317]:       <target dev="tap42a38c57-43"/>
Apr 14 04:20:39 user nova-compute[71317]:     </interface>
Apr 14 04:20:39 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:20:39 user nova-compute[71317]:       <mac address="fa:16:3e:ab:44:84"/>
Apr 14 04:20:39 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:39 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:20:39 user nova-compute[71317]:       <target dev="tap186b0001-a3"/>
Apr 14 04:20:39 user nova-compute[71317]:     </interface>
Apr 14 04:20:39 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:20:39 user nova-compute[71317]:       <mac address="fa:16:3e:27:63:d9"/>
Apr 14 04:20:39 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:39 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:20:39 user nova-compute[71317]:       <target dev="tap70e5d70d-d9"/>
Apr 14 04:20:39 user nova-compute[71317]:     </interface>
Apr 14 04:20:39 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:20:39 user nova-compute[71317]:       <mac address="fa:16:3e:7e:6a:92"/>
Apr 14 04:20:39 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:39 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:20:39 user nova-compute[71317]:       <target dev="tap8bfa317e-39"/>
Apr 14 04:20:39 user nova-compute[71317]:     </interface>
Apr 14 04:20:39 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:20:39 user nova-compute[71317]:       <mac address="fa:16:3e:f7:31:a4"/>
Apr 14 04:20:39 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:39 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:20:39 user nova-compute[71317]:       <target dev="tap97ba6e98-09"/>
Apr 14 04:20:39 user nova-compute[71317]:     </interface>
Apr 14 04:20:39 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:20:39 user nova-compute[71317]:       <mac address="fa:16:3e:b2:3f:e0"/>
Apr 14 04:20:39 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:39 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:20:39 user nova-compute[71317]:       <target dev="tap62770b99-cf"/>
Apr 14 04:20:39 user nova-compute[71317]:     </interface>
Apr 14 04:20:39 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:20:39 user nova-compute[71317]:       <mac address="fa:16:3e:06:02:5f"/>
Apr 14 04:20:39 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:39 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:20:39 user nova-compute[71317]:       <target dev="tap7249969b-00"/>
Apr 14 04:20:39 user nova-compute[71317]:     </interface>
Apr 14 04:20:39 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:20:39 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/3d9cd123-3212-4a1d-b7c8-cf0441101048/console.log" append="off"/>
Apr 14 04:20:39 user nova-compute[71317]:     </serial>
Apr 14 04:20:39 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:20:39 user nova-compute[71317]:     <video>
Apr 14 04:20:39 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:39 user nova-compute[71317]:     </video>
Apr 14 04:20:39 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:20:39 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:20:39 user nova-compute[71317]:     </rng>
Apr 14 04:20:39 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:20:39 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:20:39 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:20:39 user nova-compute[71317]:     </memballoon>
Apr 14 04:20:39 user nova-compute[71317]:   </devices>
Apr 14 04:20:39 user nova-compute[71317]: </domain>
Apr 14 04:20:39 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "42a38c57-4357-4a88-91e3-d76bb38534db", "address": "fa:16:3e:42:d6:fc", "network": {"id": "75bed293-1d30-421a-9656-15f0ead0650b", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1531253437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap42a38c57-43", "ovs_interfaceid": "42a38c57-4357-4a88-91e3-d76bb38534db", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "42a38c57-4357-4a88-91e3-d76bb38534db", "address": "fa:16:3e:42:d6:fc", "network": {"id": "75bed293-1d30-421a-9656-15f0ead0650b", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1531253437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap42a38c57-43", "ovs_interfaceid": "42a38c57-4357-4a88-91e3-d76bb38534db", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:42:d6:fc,bridge_name='br-int',has_traffic_filtering=True,id=42a38c57-4357-4a88-91e3-d76bb38534db,network=Network(75bed293-1d30-421a-9656-15f0ead0650b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap42a38c57-43') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:42:d6:fc,bridge_name='br-int',has_traffic_filtering=True,id=42a38c57-4357-4a88-91e3-d76bb38534db,network=Network(75bed293-1d30-421a-9656-15f0ead0650b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap42a38c57-43') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap42a38c57-43, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap42a38c57-43, col_values=(('external_ids', {'iface-id': '42a38c57-4357-4a88-91e3-d76bb38534db', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:42:d6:fc', 'vm-uuid': '3d9cd123-3212-4a1d-b7c8-cf0441101048'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: INFO os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:42:d6:fc,bridge_name='br-int',has_traffic_filtering=True,id=42a38c57-4357-4a88-91e3-d76bb38534db,network=Network(75bed293-1d30-421a-9656-15f0ead0650b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap42a38c57-43')
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "address": "fa:16:3e:ab:44:84", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap186b0001-a3", "ovs_interfaceid": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "address": "fa:16:3e:ab:44:84", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap186b0001-a3", "ovs_interfaceid": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ab:44:84,bridge_name='br-int',has_traffic_filtering=True,id=186b0001-a3cf-40fa-babf-6b7045fbdb26,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap186b0001-a3') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ab:44:84,bridge_name='br-int',has_traffic_filtering=True,id=186b0001-a3cf-40fa-babf-6b7045fbdb26,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap186b0001-a3') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap186b0001-a3, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap186b0001-a3, col_values=(('external_ids', {'iface-id': '186b0001-a3cf-40fa-babf-6b7045fbdb26', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ab:44:84', 'vm-uuid': '3d9cd123-3212-4a1d-b7c8-cf0441101048'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: INFO os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ab:44:84,bridge_name='br-int',has_traffic_filtering=True,id=186b0001-a3cf-40fa-babf-6b7045fbdb26,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap186b0001-a3')
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "address": "fa:16:3e:27:63:d9", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.57", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap70e5d70d-d9", "ovs_interfaceid": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "address": "fa:16:3e:27:63:d9", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.57", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap70e5d70d-d9", "ovs_interfaceid": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:27:63:d9,bridge_name='br-int',has_traffic_filtering=True,id=70e5d70d-d9af-4853-9de8-2cbff2433c60,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap70e5d70d-d9') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:27:63:d9,bridge_name='br-int',has_traffic_filtering=True,id=70e5d70d-d9af-4853-9de8-2cbff2433c60,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap70e5d70d-d9') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap70e5d70d-d9, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap70e5d70d-d9, col_values=(('external_ids', {'iface-id': '70e5d70d-d9af-4853-9de8-2cbff2433c60', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:27:63:d9', 'vm-uuid': '3d9cd123-3212-4a1d-b7c8-cf0441101048'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: INFO os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:27:63:d9,bridge_name='br-int',has_traffic_filtering=True,id=70e5d70d-d9af-4853-9de8-2cbff2433c60,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap70e5d70d-d9')
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "address": "fa:16:3e:7e:6a:92", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.37", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8bfa317e-39", "ovs_interfaceid": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "address": "fa:16:3e:7e:6a:92", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.37", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8bfa317e-39", "ovs_interfaceid": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7e:6a:92,bridge_name='br-int',has_traffic_filtering=True,id=8bfa317e-3995-442d-b5ab-cb1199cb1fb8,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8bfa317e-39') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:7e:6a:92,bridge_name='br-int',has_traffic_filtering=True,id=8bfa317e-3995-442d-b5ab-cb1199cb1fb8,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8bfa317e-39') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8bfa317e-39, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap8bfa317e-39, col_values=(('external_ids', {'iface-id': '8bfa317e-3995-442d-b5ab-cb1199cb1fb8', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:7e:6a:92', 'vm-uuid': '3d9cd123-3212-4a1d-b7c8-cf0441101048'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-ff8adc02-3e40-4aba-976a-60f9b7f91c90 req-3b3ef6b9-1d01-4ee8-9436-d208341b0679 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Received event network-vif-plugged-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ff8adc02-3e40-4aba-976a-60f9b7f91c90 req-3b3ef6b9-1d01-4ee8-9436-d208341b0679 service nova] Acquiring lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ff8adc02-3e40-4aba-976a-60f9b7f91c90 req-3b3ef6b9-1d01-4ee8-9436-d208341b0679 service nova] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ff8adc02-3e40-4aba-976a-60f9b7f91c90 req-3b3ef6b9-1d01-4ee8-9436-d208341b0679 service nova] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-ff8adc02-3e40-4aba-976a-60f9b7f91c90 req-3b3ef6b9-1d01-4ee8-9436-d208341b0679 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] No waiting events found dispatching network-vif-plugged-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:40 user nova-compute[71317]: WARNING nova.compute.manager [req-ff8adc02-3e40-4aba-976a-60f9b7f91c90 req-3b3ef6b9-1d01-4ee8-9436-d208341b0679 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Received unexpected event network-vif-plugged-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f for instance with vm_state building and task_state spawning.
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: INFO os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:7e:6a:92,bridge_name='br-int',has_traffic_filtering=True,id=8bfa317e-3995-442d-b5ab-cb1199cb1fb8,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8bfa317e-39')
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "address": "fa:16:3e:f7:31:a4", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.117", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap97ba6e98-09", "ovs_interfaceid": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "address": "fa:16:3e:f7:31:a4", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.117", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap97ba6e98-09", "ovs_interfaceid": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=97ba6e98-092c-472f-9f27-eab96fe7f9ae,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap97ba6e98-09') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=97ba6e98-092c-472f-9f27-eab96fe7f9ae,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap97ba6e98-09') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap97ba6e98-09, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap97ba6e98-09, col_values=(('external_ids', {'iface-id': '97ba6e98-092c-472f-9f27-eab96fe7f9ae', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f7:31:a4', 'vm-uuid': '3d9cd123-3212-4a1d-b7c8-cf0441101048'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: INFO os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=97ba6e98-092c-472f-9f27-eab96fe7f9ae,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap97ba6e98-09')
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "62770b99-cf54-4132-ad17-d1859ddf8443", "address": "fa:16:3e:b2:3f:e0", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap62770b99-cf", "ovs_interfaceid": "62770b99-cf54-4132-ad17-d1859ddf8443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "62770b99-cf54-4132-ad17-d1859ddf8443", "address": "fa:16:3e:b2:3f:e0", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap62770b99-cf", "ovs_interfaceid": "62770b99-cf54-4132-ad17-d1859ddf8443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b2:3f:e0,bridge_name='br-int',has_traffic_filtering=True,id=62770b99-cf54-4132-ad17-d1859ddf8443,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap62770b99-cf') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b2:3f:e0,bridge_name='br-int',has_traffic_filtering=True,id=62770b99-cf54-4132-ad17-d1859ddf8443,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap62770b99-cf') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap62770b99-cf, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap62770b99-cf, col_values=(('external_ids', {'iface-id': '62770b99-cf54-4132-ad17-d1859ddf8443', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b2:3f:e0', 'vm-uuid': '3d9cd123-3212-4a1d-b7c8-cf0441101048'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: INFO os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b2:3f:e0,bridge_name='br-int',has_traffic_filtering=True,id=62770b99-cf54-4132-ad17-d1859ddf8443,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap62770b99-cf')
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "address": "fa:16:3e:06:02:5f", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7249969b-00", "ovs_interfaceid": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "address": "fa:16:3e:06:02:5f", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7249969b-00", "ovs_interfaceid": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:06:02:5f,bridge_name='br-int',has_traffic_filtering=True,id=7249969b-00f8-4c4d-8573-832c3e0eb8fe,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7249969b-00') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:06:02:5f,bridge_name='br-int',has_traffic_filtering=True,id=7249969b-00f8-4c4d-8573-832c3e0eb8fe,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7249969b-00') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7249969b-00, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap7249969b-00, col_values=(('external_ids', {'iface-id': '7249969b-00f8-4c4d-8573-832c3e0eb8fe', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:06:02:5f', 'vm-uuid': '3d9cd123-3212-4a1d-b7c8-cf0441101048'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: INFO os_vif [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:06:02:5f,bridge_name='br-int',has_traffic_filtering=True,id=7249969b-00f8-4c4d-8573-832c3e0eb8fe,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7249969b-00')
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] No BDM found with device name hda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] No BDM found with device name vdc, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] No VIF found with MAC fa:16:3e:42:d6:fc, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] No VIF found with MAC fa:16:3e:f7:31:a4, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:20:40 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Using config drive
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:40 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Creating config drive at /opt/stack/data/nova/instances/3d9cd123-3212-4a1d-b7c8-cf0441101048/disk.config
Apr 14 04:20:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/3d9cd123-3212-4a1d-b7c8-cf0441101048/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpd5iyfc3l {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/3d9cd123-3212-4a1d-b7c8-cf0441101048/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpd5iyfc3l" returned: 0 in 0.161s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:40 user nova-compute[71317]: INFO nova.compute.manager [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Rebuilding instance
Apr 14 04:20:41 user nova-compute[71317]: DEBUG nova.objects.instance [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'trusted_certs' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG nova.objects.instance [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'pci_requests' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG nova.objects.instance [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'pci_devices' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG nova.objects.instance [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'resources' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG nova.objects.instance [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'migration_context' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG nova.objects.instance [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Trying to apply a migration context that does not seem to be set for this instance {{(pid=71317) apply_migration_context /opt/stack/nova/nova/objects/instance.py:1032}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-81b51ea7-71be-4b12-93bb-987888b7e3c0 req-eeccefb2-41ab-49f3-9032-5d134feae72c service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-unplugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81b51ea7-71be-4b12-93bb-987888b7e3c0 req-eeccefb2-41ab-49f3-9032-5d134feae72c service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81b51ea7-71be-4b12-93bb-987888b7e3c0 req-eeccefb2-41ab-49f3-9032-5d134feae72c service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81b51ea7-71be-4b12-93bb-987888b7e3c0 req-eeccefb2-41ab-49f3-9032-5d134feae72c service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-81b51ea7-71be-4b12-93bb-987888b7e3c0 req-eeccefb2-41ab-49f3-9032-5d134feae72c service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-unplugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:41 user nova-compute[71317]: WARNING nova.compute.manager [req-81b51ea7-71be-4b12-93bb-987888b7e3c0 req-eeccefb2-41ab-49f3-9032-5d134feae72c service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-unplugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state active and task_state rebuilding.
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-f60fb214-91a3-4421-b05d-5e5a6de9fdae req-05aab2d5-f942-4e04-a90d-e18fa22fbccb service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Received event network-vif-plugged-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f60fb214-91a3-4421-b05d-5e5a6de9fdae req-05aab2d5-f942-4e04-a90d-e18fa22fbccb service nova] Acquiring lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f60fb214-91a3-4421-b05d-5e5a6de9fdae req-05aab2d5-f942-4e04-a90d-e18fa22fbccb service nova] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f60fb214-91a3-4421-b05d-5e5a6de9fdae req-05aab2d5-f942-4e04-a90d-e18fa22fbccb service nova] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-f60fb214-91a3-4421-b05d-5e5a6de9fdae req-05aab2d5-f942-4e04-a90d-e18fa22fbccb service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] No waiting events found dispatching network-vif-plugged-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:42 user nova-compute[71317]: WARNING nova.compute.manager [req-f60fb214-91a3-4421-b05d-5e5a6de9fdae req-05aab2d5-f942-4e04-a90d-e18fa22fbccb service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Received unexpected event network-vif-plugged-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f for instance with vm_state building and task_state spawning.
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446042.5164173, d2e6cf32-080b-47b2-b4db-2b776ba85b38 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:42 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] VM Resumed (Lifecycle Event)
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:20:42 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Instance spawned successfully.
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SvtMEfzm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:42 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:42 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:42 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:42 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446042.525093, d2e6cf32-080b-47b2-b4db-2b776ba85b38 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:42 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] VM Started (Lifecycle Event)
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b474538-6294-430c-b2e8-f856f781f795]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:20:42 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:42 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance destroyed successfully.
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1a0ed62-12cf-4973-9c9d-62fc26fc6710]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5] added
Apr 14 04:20:42 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[53f05eaa-6208-4d33-b14d-879e1d7eb500]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:42 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b12e50f-f6cf-4106-85e5-09487a5e9fb4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SvtMEfzm {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SvtMEfzm" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e75ec3b-0a3e-45fd-a8fa-f688c535e966]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'SvtMEfzm'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:42 user nova-compute[71317]: INFO nova.compute.manager [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Took 8.13 seconds to spawn the instance on the hypervisor.
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a635dc06-a199-4077-a56f-70e4dc0ee81f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:42 user nova-compute[71317]: INFO nova.compute.manager [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Detaching volume 3e0082ad-1ad0-4980-a066-6017f65fce9e
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f154d1b8-c2db-4557-8146-2ba308c0340a]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:42 user nova-compute[71317]: INFO nova.compute.manager [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Took 10.01 seconds to build instance.
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ea87d5ee-2625-447d-91ad-9bf7c006290f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.238s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:42 user nova-compute[71317]: INFO nova.virt.block_device [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Attempting to driver detach volume 3e0082ad-1ad0-4980-a066-6017f65fce9e from mountpoint /dev/vdb
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --login" returned: 0 in 0.093s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9e41e6f-0d4f-4da4-b3ba-ad6cbc4491eb]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5, portal: 172.16.0.220,3260]
Apr 14 04:20:42 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5, portal: 172.16.0.220,3260] successful.
Apr 14 04:20:42 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Attempting to detach device vdb from instance a0a20aeb-ed06-4c65-b87d-74f8f825457d from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:20:42 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:20:42 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:20:42 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:20:42 user nova-compute[71317]:   <serial>3e0082ad-1ad0-4980-a066-6017f65fce9e</serial>
Apr 14 04:20:42 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:20:42 user nova-compute[71317]: </disk>
Apr 14 04:20:42 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee266803-c6b4-4eb0-975d-c21bad651206]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:42 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully detached device vdb from instance a0a20aeb-ed06-4c65-b87d-74f8f825457d from the persistent domain config.
Apr 14 04:20:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YjWqRKtu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:42 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[71a797c4-2c30-4a39-82cd-4dac91278df9]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:20:42 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.381s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] HCTL ('39', '-', '-', 0) found on session 19 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.040s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[27c08c43-c9e4-4e4d-92bb-f0deafdd6c0a]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Searching for a device in session 19 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Connected to sdh using {'target_iqn': 'iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SvtMEfzm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-7a4331ef-0802-4d10-89bc-3fa78f4aee0d req-4975f243-3496-4131-a119-50a6aaae119e service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-186b0001-a3cf-40fa-babf-6b7045fbdb26 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7a4331ef-0802-4d10-89bc-3fa78f4aee0d req-4975f243-3496-4131-a119-50a6aaae119e service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7a4331ef-0802-4d10-89bc-3fa78f4aee0d req-4975f243-3496-4131-a119-50a6aaae119e service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7a4331ef-0802-4d10-89bc-3fa78f4aee0d req-4975f243-3496-4131-a119-50a6aaae119e service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-7a4331ef-0802-4d10-89bc-3fa78f4aee0d req-4975f243-3496-4131-a119-50a6aaae119e service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-186b0001-a3cf-40fa-babf-6b7045fbdb26 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:43 user nova-compute[71317]: WARNING nova.compute.manager [req-7a4331ef-0802-4d10-89bc-3fa78f4aee0d req-4975f243-3496-4131-a119-50a6aaae119e service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-186b0001-a3cf-40fa-babf-6b7045fbdb26 for instance with vm_state building and task_state spawning.
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:43 user nova-compute[71317]: WARNING nova.compute.manager [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state active and task_state rebuilding.
Apr 14 04:20:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:43 user nova-compute[71317]: WARNING nova.compute.manager [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-42a38c57-4357-4a88-91e3-d76bb38534db for instance with vm_state building and task_state spawning.
Apr 14 04:20:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:43 user nova-compute[71317]: WARNING nova.compute.manager [req-842f8bb0-9a37-471c-b2be-c36bb319e63e req-7de74eac-c717-4813-a01e-83ba4dfc1663 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-42a38c57-4357-4a88-91e3-d76bb38534db for instance with vm_state building and task_state spawning.
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446028.8943677, b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:43 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] VM Stopped (Lifecycle Event)
Apr 14 04:20:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ea22f77e-7b6f-4284-8e42-87c934ea669a None None] [instance: b0d74dbd-d61f-4132-9e0b-ec534ec2f6a7] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.454s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] <== connect_volume: return (1455ms) {'type': 'block', 'scsi_wwn': '23166346461363135', 'path': '/dev/sdh'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23166346461363135', 'path': '/dev/sdh'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 1.082s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'flavor' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.036s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[701bae1d-959b-461b-a9b5-799776744b86]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:20:44 user nova-compute[71317]: No targets found.
Apr 14 04:20:44 user nova-compute[71317]: iSNS:
Apr 14 04:20:44 user nova-compute[71317]: No targets found.
Apr 14 04:20:44 user nova-compute[71317]: STATIC:
Apr 14 04:20:44 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5
Apr 14 04:20:44 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:44 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:44 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e
Apr 14 04:20:44 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:44 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:44 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94
Apr 14 04:20:44 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:44 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:44 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3
Apr 14 04:20:44 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:44 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:44 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac
Apr 14 04:20:44 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:44 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:44 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497
Apr 14 04:20:44 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:44 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:44 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:20:44 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:44 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:44 user nova-compute[71317]: FIRMWARE:
Apr 14 04:20:44 user nova-compute[71317]: No targets found.
Apr 14 04:20:44 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:20:44 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:20:44 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:20:44 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:20:44 user nova-compute[71317]:   <source dev="/dev/sdh"/>
Apr 14 04:20:44 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:20:44 user nova-compute[71317]:   <serial>3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5</serial>
Apr 14 04:20:44 user nova-compute[71317]: </disk>
Apr 14 04:20:44 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[26a90011-8267-4828-ab5b-ece434d0ef7f]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[99662e0c-8793-49a4-b64c-c7954ea9460c]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:44 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:44 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:44 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:44 user nova-compute[71317]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:44 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:44 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:20:44 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:44 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:44 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:44 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:44 user nova-compute[71317]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:44 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:44 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:20:44 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d180bb50>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e'): ({'sdf'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Removing single pathed devices sdf {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.046s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[43050b90-f0f3-4a57-ae56-70425c432813]: (4, ('path checker states:\nup                  6\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[add39905-7af8-4e39-ac8f-192fb8809700]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flushing IO for device /dev/sdf {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.033s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d0cc139-672a-4d1b-b15f-5842f2663233]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.036s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e2b105d-25b3-41ad-9a20-6746424fe8e1]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4e9e7ca-2f1f-4ad0-9660-f7a588204a5c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --logout" returned: 0 in 0.084s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8995ca5-1245-4659-aa84-f8f7f949887a]: (4, ('Logging out of session [sid: 17, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260]\nLogout of [sid: 17, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 17, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260]
Apr 14 04:20:44 user nova-compute[71317]: Logout of [sid: 17, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260] successful.
Apr 14 04:20:44 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op delete" returned: 0 in 0.034s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6619be3b-c422-476b-9be6-c8b2212c902a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.462s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== disconnect_volume: return (1545ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] No VIF found with MAC fa:16:3e:73:25:41, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5ed43b12-12d0-4bd1-8c4a-d85304d8d0eb tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 8.201s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance destroyed successfully.
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1350614351',display_name='tempest-ServerActionsTestJSON-server-1398117926',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1350614351',id=6,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:01Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-789bnjb3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='rebuilding',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=a0a20aeb-ed06-4c65-b87d-74f8f825457d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_vif [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3a3e3b13-7b, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: INFO os_vif [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b')
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YjWqRKtu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:45 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.053s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f484fdd0-8068-4eac-b7bd-75d1c3a1c242]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:20:45 user nova-compute[71317]: No targets found.
Apr 14 04:20:45 user nova-compute[71317]: iSNS:
Apr 14 04:20:45 user nova-compute[71317]: No targets found.
Apr 14 04:20:45 user nova-compute[71317]: STATIC:
Apr 14 04:20:45 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5
Apr 14 04:20:45 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:45 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:45 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94
Apr 14 04:20:45 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:45 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:45 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3
Apr 14 04:20:45 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:45 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:45 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac
Apr 14 04:20:45 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:45 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:45 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497
Apr 14 04:20:45 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:45 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:45 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:20:45 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:20:45 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:20:45 user nova-compute[71317]: FIRMWARE:
Apr 14 04:20:45 user nova-compute[71317]: No targets found.
Apr 14 04:20:45 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:20:45 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:20:45 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac419835-d3e3-41d0-8562-8a56c854bc1e]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.045s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fbd4bc36-ddd5-4571-b051-9093cf6b9b4c]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:45 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:45 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:45 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:45 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:45 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:20:45 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:45 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:45 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:45 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:45 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:45 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:20:45 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84b42167a0>, {}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Disconnecting from: [] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.153s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== disconnect_volume: return (155ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:20:45 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Deleting instance files /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d_del
Apr 14 04:20:45 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Deletion of /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d_del complete
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-186b0001-a3cf-40fa-babf-6b7045fbdb26 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-186b0001-a3cf-40fa-babf-6b7045fbdb26 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:45 user nova-compute[71317]: WARNING nova.compute.manager [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-186b0001-a3cf-40fa-babf-6b7045fbdb26 for instance with vm_state building and task_state spawning.
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-70e5d70d-d9af-4853-9de8-2cbff2433c60 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-70e5d70d-d9af-4853-9de8-2cbff2433c60 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:45 user nova-compute[71317]: WARNING nova.compute.manager [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-70e5d70d-d9af-4853-9de8-2cbff2433c60 for instance with vm_state building and task_state spawning.
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-70e5d70d-d9af-4853-9de8-2cbff2433c60 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-70e5d70d-d9af-4853-9de8-2cbff2433c60 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:45 user nova-compute[71317]: WARNING nova.compute.manager [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-70e5d70d-d9af-4853-9de8-2cbff2433c60 for instance with vm_state building and task_state spawning.
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:45 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:45 user nova-compute[71317]: WARNING nova.compute.manager [req-2a716b6c-d063-451d-90bd-a3e8728a0345 req-29839009-922c-4c12-bf0d-9e7a72c0c062 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 for instance with vm_state building and task_state spawning.
Apr 14 04:20:45 user nova-compute[71317]: INFO nova.virt.block_device [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Booting with volume 3e0082ad-1ad0-4980-a066-6017f65fce9e at /dev/vdb
Apr 14 04:20:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:46 user nova-compute[71317]: WARNING nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-97ba6e98-092c-472f-9f27-eab96fe7f9ae for instance with vm_state building and task_state spawning.
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:46 user nova-compute[71317]: WARNING nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-97ba6e98-092c-472f-9f27-eab96fe7f9ae for instance with vm_state building and task_state spawning.
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:46 user nova-compute[71317]: WARNING nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-62770b99-cf54-4132-ad17-d1859ddf8443 for instance with vm_state building and task_state spawning.
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:46 user nova-compute[71317]: WARNING nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-62770b99-cf54-4132-ad17-d1859ddf8443 for instance with vm_state building and task_state spawning.
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:46 user nova-compute[71317]: WARNING nova.compute.manager [req-ec8a393f-58b7-4a7c-b034-cb3323fd338e req-929a11cf-abbb-4296-83da-c66e437d5cad service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-7249969b-00f8-4c4d-8573-832c3e0eb8fe for instance with vm_state building and task_state spawning.
Apr 14 04:20:46 user nova-compute[71317]: DEBUG os_brick.utils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a44d4f33-9813-4cec-bf09-5379da067778]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[16cdcbbb-f653-4cc6-8105-a3396ab1458f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[484657a8-1da2-4b84-a205-dc7227fa9b70]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bfb85521-84ca-4abf-a360-4d8f331ea97c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:20:46 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[78c3548e-08e8-44b3-b2a2-83174532777a]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG os_brick.utils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== get_connector_properties: return (174ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating existing volume attachment record: 5257d9f6-4ed2-4452-88cf-1cd5a4131e1f {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:20:46 user nova-compute[71317]: INFO nova.compute.manager [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Rescuing
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquired lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.network.neutron [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG nova.network.neutron [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updating instance_info_cache with network_info: [{"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Releasing lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:47 user nova-compute[71317]: WARNING nova.compute.manager [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 for instance with vm_state building and task_state spawning.
Apr 14 04:20:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-unplugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-unplugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:47 user nova-compute[71317]: WARNING nova.compute.manager [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-unplugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state active and task_state rescuing.
Apr 14 04:20:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:47 user nova-compute[71317]: WARNING nova.compute.manager [req-96a31d5f-250b-49ad-8c27-fd92b88f153a req-7fc8c820-38f2-4aaa-bb9f-8a7f9b4ef785 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state active and task_state rescuing.
Apr 14 04:20:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:47 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Instance destroyed successfully.
Apr 14 04:20:47 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Attempting a stable device rescue
Apr 14 04:20:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} {{(pid=71317) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Instance directory exists: not creating {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 04:20:48 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Creating image(s)
Apr 14 04:20:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "/opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "/opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "/opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'trusted_certs' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "7a87dc15b3f1a9b23f4facddd2d5d74578249812" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "7a87dc15b3f1a9b23f4facddd2d5d74578249812" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446048.074443, 3d9cd123-3212-4a1d-b7c8-cf0441101048 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:48 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] VM Resumed (Lifecycle Event)
Apr 14 04:20:48 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Instance spawned successfully.
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-432b49eb-d735-4ce2-9981-39072f7ec358 req-2251e344-f289-486d-a4c2-3ca111fab785 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-432b49eb-d735-4ce2-9981-39072f7ec358 req-2251e344-f289-486d-a4c2-3ca111fab785 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-432b49eb-d735-4ce2-9981-39072f7ec358 req-2251e344-f289-486d-a4c2-3ca111fab785 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-432b49eb-d735-4ce2-9981-39072f7ec358 req-2251e344-f289-486d-a4c2-3ca111fab785 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-432b49eb-d735-4ce2-9981-39072f7ec358 req-2251e344-f289-486d-a4c2-3ca111fab785 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:48 user nova-compute[71317]: WARNING nova.compute.manager [req-432b49eb-d735-4ce2-9981-39072f7ec358 req-2251e344-f289-486d-a4c2-3ca111fab785 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-7249969b-00f8-4c4d-8573-832c3e0eb8fe for instance with vm_state building and task_state spawning.
Apr 14 04:20:48 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446048.0746553, 3d9cd123-3212-4a1d-b7c8-cf0441101048 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:48 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] VM Started (Lifecycle Event)
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:20:48 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:20:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812.part --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:49 user nova-compute[71317]: INFO nova.compute.manager [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Took 16.18 seconds to spawn the instance on the hypervisor.
Apr 14 04:20:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812.part --force-share --output=json" returned: 0 in 0.172s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG nova.virt.images [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] 91fd17de-3c0e-4dfd-bb7c-1f98b14aaeb8 was qcow2, converting to raw {{(pid=71317) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812.part /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812.converted {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:49 user nova-compute[71317]: INFO nova.compute.manager [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Took 33.71 seconds to build instance.
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fa66ca35-638b-4ad2-a9b7-15c35d09b774 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 33.877s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812.part /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812.converted" returned: 0 in 0.547s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812.converted --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812.converted --force-share --output=json" returned: 0 in 0.146s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "7a87dc15b3f1a9b23f4facddd2d5d74578249812" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.767s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "7a87dc15b3f1a9b23f4facddd2d5d74578249812" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "7a87dc15b3f1a9b23f4facddd2d5d74578249812" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:49 user nova-compute[71317]: WARNING nova.compute.manager [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state active and task_state rescuing.
Apr 14 04:20:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:49 user nova-compute[71317]: WARNING nova.compute.manager [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state active and task_state rescuing.
Apr 14 04:20:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-unplugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-unplugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:49 user nova-compute[71317]: WARNING nova.compute.manager [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-unplugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state active and task_state rescuing.
Apr 14 04:20:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:49 user nova-compute[71317]: WARNING nova.compute.manager [req-592f97de-66ec-4d0f-83f4-8a7cd012e79b req-928af76e-6e77-471c-a5d1-25824029346c service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state active and task_state rescuing.
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812 --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812,backing_fmt=raw /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.rescue {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812,backing_fmt=raw /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.rescue" returned: 0 in 0.048s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "7a87dc15b3f1a9b23f4facddd2d5d74578249812" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.203s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'migration_context' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Start _get_guest_xml network_info=[{"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "vif_mac": "fa:16:3e:73:25:41"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '91fd17de-3c0e-4dfd-bb7c-1f98b14aaeb8', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': None, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SvtMEfzm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, 'status': 'reserved', 'instance': 'cd60eede-cd79-4c1b-adb7-ba7c6cbc9659', 'attached_at': '', 'detached_at': '', 'volume_id': '3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5', 'serial': '3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5'}, 'attachment_id': 'f6c08d20-8171-4b4a-a116-07bc4dc24f0c', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'resources' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'numa_topology' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:50 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:20:50 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'vcpu_model' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SvtMEfzm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:50 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:50 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:50 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ab6d7e8-80dc-4b5c-a64b-104a6ee77518]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = SvtMEfzm\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Apr 14 04:20:50 user nova-compute[71317]: node.name = iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5
Apr 14 04:20:50 user nova-compute[71317]: node.tpgt = -1
Apr 14 04:20:50 user nova-compute[71317]: node.startup = automatic
Apr 14 04:20:50 user nova-compute[71317]: node.leading_login = No
Apr 14 04:20:50 user nova-compute[71317]: iface.iscsi_ifacename = default
Apr 14 04:20:50 user nova-compute[71317]: iface.net_ifacename = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.ipaddress = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.prefix_len = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.hwaddress = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.transport_name = tcp
Apr 14 04:20:50 user nova-compute[71317]: iface.initiatorname = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.state = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.vlan_id = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.vlan_priority = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.vlan_state = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.iface_num = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.mtu = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.port = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.bootproto = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.subnet_mask = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.gateway = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.dhcp_alt_client_id_state = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.dhcp_alt_client_id = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.dhcp_dns = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.dhcp_learn_iqn = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.dhcp_req_vendor_id_state = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.dhcp_vendor_id_state = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.dhcp_vendor_id = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.dhcp_slp_da = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.fragmentation = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.gratuitous_arp = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.incoming_forwarding = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.tos_state = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.tos = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.ttl = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.delayed_ack = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.tcp_nagle = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.tcp_wsf_state = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.tcp_wsf = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.tcp_timer_scale = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.tcp_timestamp = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.redirect = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.def_task_mgmt_timeout = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.header_digest = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.data_digest = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.immediate_data = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.initial_r2t = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.data_seq_inorder = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.data_pdu_inorder = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.erl = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.max_receive_data_len = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.first_burst_len = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.max_outstanding_r2t = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.max_burst_len = 0
Apr 14 04:20:50 user nova-compute[71317]: iface.chap_auth = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.bidi_chap = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.strict_login_compliance = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.discovery_auth = <empty>
Apr 14 04:20:50 user nova-compute[71317]: iface.discovery_logout = <empty>
Apr 14 04:20:50 user nova-compute[71317]: node.discovery_address = <empty>
Apr 14 04:20:50 user nova-compute[71317]: node.discovery_port = 0
Apr 14 04:20:50 user nova-compute[71317]: node.discovery_type = static
Apr 14 04:20:50 user nova-compute[71317]: node.session.initial_cmdsn = 0
Apr 14 04:20:50 user nova-compute[71317]: node.session.initial_login_retry_max = 8
Apr 14 04:20:50 user nova-compute[71317]: node.session.xmit_thread_priority = -20
Apr 14 04:20:50 user nova-compute[71317]: node.session.cmds_max = 128
Apr 14 04:20:50 user nova-compute[71317]: node.session.queue_depth = 32
Apr 14 04:20:50 user nova-compute[71317]: node.session.nr_sessions = 1
Apr 14 04:20:50 user nova-compute[71317]: node.session.auth.authmethod = CHAP
Apr 14 04:20:50 user nova-compute[71317]: node.session.auth.username = SvtMEfzm
Apr 14 04:20:50 user nova-compute[71317]: node.session.auth.password = ***
Apr 14 04:20:50 user nova-compute[71317]: node.session.auth.username_in = <empty>
Apr 14 04:20:50 user nova-compute[71317]: node.session.auth.password_in = <empty>
Apr 14 04:20:50 user nova-compute[71317]: node.session.auth.chap_algs = MD5
Apr 14 04:20:50 user nova-compute[71317]: node.session.timeo.replacement_timeout = 120
Apr 14 04:20:50 user nova-compute[71317]: node.session.err_timeo.abort_timeout = 15
Apr 14 04:20:50 user nova-compute[71317]: node.session.err_timeo.lu_reset_timeout = 30
Apr 14 04:20:50 user nova-compute[71317]: node.session.err_timeo.tgt_reset_timeout = 30
Apr 14 04:20:50 user nova-compute[71317]: node.session.err_timeo.host_reset_timeout = 60
Apr 14 04:20:50 user nova-compute[71317]: node.session.iscsi.FastAbort = Yes
Apr 14 04:20:50 user nova-compute[71317]: node.session.iscsi.InitialR2T = No
Apr 14 04:20:50 user nova-compute[71317]: node.session.iscsi.ImmediateData = Yes
Apr 14 04:20:50 user nova-compute[71317]: node.session.iscsi.FirstBurstLength = 262144
Apr 14 04:20:50 user nova-compute[71317]: node.session.iscsi.MaxBurstLength = 16776192
Apr 14 04:20:50 user nova-compute[71317]: node.session.iscsi.DefaultTime2Retain = 0
Apr 14 04:20:50 user nova-compute[71317]: node.session.iscsi.DefaultTime2Wait = 2
Apr 14 04:20:50 user nova-compute[71317]: node.session.iscsi.MaxConnections = 1
Apr 14 04:20:50 user nova-compute[71317]: node.session.iscsi.MaxOutstandingR2T = 1
Apr 14 04:20:50 user nova-compute[71317]: node.session.iscsi.ERL = 0
Apr 14 04:20:50 user nova-compute[71317]: node.session.scan = manual
Apr 14 04:20:50 user nova-compute[71317]: node.session.reopen_max = 0
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].address = 172.16.0.220
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].port = 3260
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].startup = manual
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].tcp.window_size = 524288
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].tcp.type_of_service = 0
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].timeo.logout_timeout = 15
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].timeo.login_timeout = 15
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].timeo.auth_timeout = 45
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].timeo.noop_out_interval = 5
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].timeo.noop_out_timeout = 5
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].iscsi.HeaderDigest = None
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].iscsi.DataDigest = None
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].iscsi.IFMarker = No
Apr 14 04:20:50 user nova-compute[71317]: node.conn[0].iscsi.OFMarker = No
Apr 14 04:20:50 user nova-compute[71317]: # END RECORD
Apr 14 04:20:50 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[268addd3-c589-447f-8af3-ed0437ef088a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[203375b6-83f8-434d-ad2f-f101b40e0f13]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SvtMEfzm {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SvtMEfzm" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[78c51232-fff8-44ec-b592-b1ac84bbecc8]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'SvtMEfzm'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3969ed48-d060-4871-a816-9558ab6950cf]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1fe7e96d-b348-4154-87e2-8048ba6ed25b]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:50 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:50 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:50 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:50 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:50 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:20:50 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:50 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:50 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:50 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:50 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:50 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:20:50 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.158s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] HCTL ('39', '0', '0', 0) found on session 19 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Scanning host 39 c: 0, t: 0, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[57f2fcc8-2efa-45fa-9e86-e0d574b32797]: (4, ('0 0 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Searching for a device in session 19 and hctl ('39', '0', '0', 0) yield: sdh {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Connected to sdh using {'target_iqn': 'iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SvtMEfzm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.182s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] <== connect_volume: return (184ms) {'type': 'block', 'scsi_wwn': '23166346461363135', 'path': '/dev/sdh'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23166346461363135', 'path': '/dev/sdh'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-1335768391',display_name='tempest-ServerStableDeviceRescueTest-server-1335768391',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-1335768391',id=8,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHsERCJfV5vlojYp6CtYNDuskD1mPJwApYPpyopwqnFEg3bukLlc5MC0Mx5e6kcUmmhflLvGI9V/TTXxf//CKjD9nOwYO+5pKfSRGQB3deK57Z7HLLcWkH9v7l2Z+jP7Gg==',key_name='tempest-keypair-234703296',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:06Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='15bb9ddec81e4f6cbceb6ce7dee9c3a1',ramdisk_id='',reservation_id='r-9gfjdvk7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerStableDeviceRescueTest-1018370864',owner_user_name='tempest-ServerStableDeviceRescueTest-1018370864-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:18:46Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='676d80d3684045298d895555247174af',uuid=cd60eede-cd79-4c1b-adb7-ba7c6cbc9659,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "vif_mac": "fa:16:3e:73:25:41"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Converting VIF {"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "vif_mac": "fa:16:3e:73:25:41"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:73:25:41,bridge_name='br-int',has_traffic_filtering=True,id=0c62956f-a2d8-4932-b48f-469eba99b8e6,network=Network(97d3cce1-27a3-4e76-9d3f-44860a2a8dcf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0c62956f-a2') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'pci_devices' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:20:50 user nova-compute[71317]:   <uuid>cd60eede-cd79-4c1b-adb7-ba7c6cbc9659</uuid>
Apr 14 04:20:50 user nova-compute[71317]:   <name>instance-00000008</name>
Apr 14 04:20:50 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:20:50 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:20:50 user nova-compute[71317]:   <metadata>
Apr 14 04:20:50 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:20:50 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:20:50 user nova-compute[71317]:       <nova:name>tempest-ServerStableDeviceRescueTest-server-1335768391</nova:name>
Apr 14 04:20:50 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:20:50</nova:creationTime>
Apr 14 04:20:50 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:20:50 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:20:50 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:20:50 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:20:50 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:20:50 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:20:50 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:20:50 user nova-compute[71317]:       <nova:owner>
Apr 14 04:20:50 user nova-compute[71317]:         <nova:user uuid="676d80d3684045298d895555247174af">tempest-ServerStableDeviceRescueTest-1018370864-project-member</nova:user>
Apr 14 04:20:50 user nova-compute[71317]:         <nova:project uuid="15bb9ddec81e4f6cbceb6ce7dee9c3a1">tempest-ServerStableDeviceRescueTest-1018370864</nova:project>
Apr 14 04:20:50 user nova-compute[71317]:       </nova:owner>
Apr 14 04:20:50 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:20:50 user nova-compute[71317]:       <nova:ports>
Apr 14 04:20:50 user nova-compute[71317]:         <nova:port uuid="0c62956f-a2d8-4932-b48f-469eba99b8e6">
Apr 14 04:20:50 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 04:20:50 user nova-compute[71317]:         </nova:port>
Apr 14 04:20:50 user nova-compute[71317]:       </nova:ports>
Apr 14 04:20:50 user nova-compute[71317]:     </nova:instance>
Apr 14 04:20:50 user nova-compute[71317]:   </metadata>
Apr 14 04:20:50 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:20:50 user nova-compute[71317]:     <system>
Apr 14 04:20:50 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:20:50 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:20:50 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:20:50 user nova-compute[71317]:       <entry name="serial">cd60eede-cd79-4c1b-adb7-ba7c6cbc9659</entry>
Apr 14 04:20:50 user nova-compute[71317]:       <entry name="uuid">cd60eede-cd79-4c1b-adb7-ba7c6cbc9659</entry>
Apr 14 04:20:50 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:20:50 user nova-compute[71317]:     </system>
Apr 14 04:20:50 user nova-compute[71317]:   </sysinfo>
Apr 14 04:20:50 user nova-compute[71317]:   <os>
Apr 14 04:20:50 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:20:50 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:20:50 user nova-compute[71317]:   </os>
Apr 14 04:20:50 user nova-compute[71317]:   <features>
Apr 14 04:20:50 user nova-compute[71317]:     <acpi/>
Apr 14 04:20:50 user nova-compute[71317]:     <apic/>
Apr 14 04:20:50 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:20:50 user nova-compute[71317]:   </features>
Apr 14 04:20:50 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:20:50 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:20:50 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:20:50 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:20:50 user nova-compute[71317]:   </clock>
Apr 14 04:20:50 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:20:50 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:20:50 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:20:50 user nova-compute[71317]:   </cpu>
Apr 14 04:20:50 user nova-compute[71317]:   <devices>
Apr 14 04:20:50 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:20:50 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:20:50 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk"/>
Apr 14 04:20:50 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:20:50 user nova-compute[71317]:     </disk>
Apr 14 04:20:50 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:20:50 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:20:50 user nova-compute[71317]:       <source dev="/dev/sdh"/>
Apr 14 04:20:50 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:20:50 user nova-compute[71317]:       <serial>3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5</serial>
Apr 14 04:20:50 user nova-compute[71317]:     </disk>
Apr 14 04:20:50 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:20:50 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:20:50 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.rescue"/>
Apr 14 04:20:50 user nova-compute[71317]:       <target dev="vdc" bus="virtio"/>
Apr 14 04:20:50 user nova-compute[71317]:       <boot order="1"/>
Apr 14 04:20:50 user nova-compute[71317]:     </disk>
Apr 14 04:20:50 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:20:50 user nova-compute[71317]:       <mac address="fa:16:3e:73:25:41"/>
Apr 14 04:20:50 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:50 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:20:50 user nova-compute[71317]:       <target dev="tap0c62956f-a2"/>
Apr 14 04:20:50 user nova-compute[71317]:     </interface>
Apr 14 04:20:50 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:20:50 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/console.log" append="off"/>
Apr 14 04:20:50 user nova-compute[71317]:     </serial>
Apr 14 04:20:50 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:20:50 user nova-compute[71317]:     <video>
Apr 14 04:20:50 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:50 user nova-compute[71317]:     </video>
Apr 14 04:20:50 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:20:50 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:20:50 user nova-compute[71317]:     </rng>
Apr 14 04:20:50 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:20:50 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:20:50 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:20:50 user nova-compute[71317]:     </memballoon>
Apr 14 04:20:50 user nova-compute[71317]:   </devices>
Apr 14 04:20:50 user nova-compute[71317]: </domain>
Apr 14 04:20:50 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:20:50 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Instance destroyed successfully.
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] No BDM found with device name vdc, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] No VIF found with MAC fa:16:3e:73:25:41, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:20:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:20:51 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Creating image(s)
Apr 14 04:20:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.151s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.151s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk 1073741824" returned: 0 in 0.057s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.214s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.175s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Checking if we can resize image /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-f9785589-26ea-4e09-8ac3-529a799f68b6 req-dc082257-de0c-4d2c-8d01-eebedb8fabff service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f9785589-26ea-4e09-8ac3-529a799f68b6 req-dc082257-de0c-4d2c-8d01-eebedb8fabff service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f9785589-26ea-4e09-8ac3-529a799f68b6 req-dc082257-de0c-4d2c-8d01-eebedb8fabff service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f9785589-26ea-4e09-8ac3-529a799f68b6 req-dc082257-de0c-4d2c-8d01-eebedb8fabff service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-f9785589-26ea-4e09-8ac3-529a799f68b6 req-dc082257-de0c-4d2c-8d01-eebedb8fabff service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:52 user nova-compute[71317]: WARNING nova.compute.manager [req-f9785589-26ea-4e09-8ac3-529a799f68b6 req-dc082257-de0c-4d2c-8d01-eebedb8fabff service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state active and task_state rescuing.
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.257s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Cannot resize image /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Ensure instance console log exists: /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Start _get_guest_xml network_info=[{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': None, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YjWqRKtu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'a0a20aeb-ed06-4c65-b87d-74f8f825457d', 'attached_at': '', 'detached_at': '', 'volume_id': '3e0082ad-1ad0-4980-a066-6017f65fce9e', 'serial': '3e0082ad-1ad0-4980-a066-6017f65fce9e'}, 'attachment_id': '5257d9f6-4ed2-4452-88cf-1cd5a4131e1f', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:20:52 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Apr 14 04:20:52 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.objects.instance [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'vcpu_model' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YjWqRKtu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:20:52 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:52 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:20:52 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:20:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260" returned: 21 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6dcc0942-949d-42d5-93a0-c5f8971e2937]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:20:52 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[00411e2a-8d00-4c1b-b2fa-0026fcc78f00]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e] added
Apr 14 04:20:52 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1420d16-19d6-404f-a623-743c638066cf]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb89cc46-a58b-45d5-bf3f-41780273d1af]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YjWqRKtu {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YjWqRKtu" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ba05fee-7c98-4449-9312-c8fc70dfa37b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YjWqRKtu'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.038s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[34bd2b6a-f90b-402b-a0b3-b8ed522e66f9]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.040s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c375a9f-ad59-4314-9c0f-dc1717badf95]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:52 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:52 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:52 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:52 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:52 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:20:52 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:52 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:52 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:52 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:52 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:52 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:20:52 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --login" returned: 0 in 0.110s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd9973de-b4b1-445d-9a15-93e0e216cd6b]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260]
Apr 14 04:20:53 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260] successful.
Apr 14 04:20:53 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[46f6116b-5bb3-4fe1-8bc5-75339ef6cd24]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.036s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f53debf-d50e-423c-99b4-20533e0f3b09]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:53 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:53 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:53 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:53 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:53 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:20:53 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:53 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:20:53 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:20:53 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:20:53 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:20:53 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:20:53 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:20:53 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:20:53 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.473s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] HCTL ('37', '-', '-', 0) found on session 20 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[648f2e8b-8ffd-4511-93f8-cdd69154a28c]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Searching for a device in session 20 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:20:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YjWqRKtu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.515s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== connect_volume: return (1517ms) {'type': 'block', 'scsi_wwn': '23362346236663531', 'path': '/dev/sdf'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23362346236663531', 'path': '/dev/sdf'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1350614351',display_name='tempest-ServerActionsTestJSON-server-1398117926',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1350614351',id=6,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:01Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-789bnjb3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:46Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=a0a20aeb-ed06-4c65-b87d-74f8f825457d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:20:54 user nova-compute[71317]:   <uuid>a0a20aeb-ed06-4c65-b87d-74f8f825457d</uuid>
Apr 14 04:20:54 user nova-compute[71317]:   <name>instance-00000006</name>
Apr 14 04:20:54 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:20:54 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:20:54 user nova-compute[71317]:   <metadata>
Apr 14 04:20:54 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:20:54 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:20:54 user nova-compute[71317]:       <nova:name>tempest-ServerActionsTestJSON-server-1398117926</nova:name>
Apr 14 04:20:54 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:20:52</nova:creationTime>
Apr 14 04:20:54 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:20:54 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:20:54 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:20:54 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:20:54 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:20:54 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:20:54 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:20:54 user nova-compute[71317]:       <nova:owner>
Apr 14 04:20:54 user nova-compute[71317]:         <nova:user uuid="e35b3ae371cf4dd4b21e2999e58cd2f4">tempest-ServerActionsTestJSON-1048786641-project-member</nova:user>
Apr 14 04:20:54 user nova-compute[71317]:         <nova:project uuid="8c14c332dfcc44b49ddf3cb482dd0d4c">tempest-ServerActionsTestJSON-1048786641</nova:project>
Apr 14 04:20:54 user nova-compute[71317]:       </nova:owner>
Apr 14 04:20:54 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:20:54 user nova-compute[71317]:       <nova:ports>
Apr 14 04:20:54 user nova-compute[71317]:         <nova:port uuid="3a3e3b13-7b4e-46c8-9123-5c1f121a38b9">
Apr 14 04:20:54 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 04:20:54 user nova-compute[71317]:         </nova:port>
Apr 14 04:20:54 user nova-compute[71317]:       </nova:ports>
Apr 14 04:20:54 user nova-compute[71317]:     </nova:instance>
Apr 14 04:20:54 user nova-compute[71317]:   </metadata>
Apr 14 04:20:54 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:20:54 user nova-compute[71317]:     <system>
Apr 14 04:20:54 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:20:54 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:20:54 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:20:54 user nova-compute[71317]:       <entry name="serial">a0a20aeb-ed06-4c65-b87d-74f8f825457d</entry>
Apr 14 04:20:54 user nova-compute[71317]:       <entry name="uuid">a0a20aeb-ed06-4c65-b87d-74f8f825457d</entry>
Apr 14 04:20:54 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:20:54 user nova-compute[71317]:     </system>
Apr 14 04:20:54 user nova-compute[71317]:   </sysinfo>
Apr 14 04:20:54 user nova-compute[71317]:   <os>
Apr 14 04:20:54 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:20:54 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:20:54 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:20:54 user nova-compute[71317]:   </os>
Apr 14 04:20:54 user nova-compute[71317]:   <features>
Apr 14 04:20:54 user nova-compute[71317]:     <acpi/>
Apr 14 04:20:54 user nova-compute[71317]:     <apic/>
Apr 14 04:20:54 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:20:54 user nova-compute[71317]:   </features>
Apr 14 04:20:54 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:20:54 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:20:54 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:20:54 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:20:54 user nova-compute[71317]:   </clock>
Apr 14 04:20:54 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:20:54 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:20:54 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:20:54 user nova-compute[71317]:   </cpu>
Apr 14 04:20:54 user nova-compute[71317]:   <devices>
Apr 14 04:20:54 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:20:54 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:20:54 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk"/>
Apr 14 04:20:54 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:20:54 user nova-compute[71317]:     </disk>
Apr 14 04:20:54 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:20:54 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:20:54 user nova-compute[71317]:       <source dev="/dev/sdf"/>
Apr 14 04:20:54 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:20:54 user nova-compute[71317]:       <serial>3e0082ad-1ad0-4980-a066-6017f65fce9e</serial>
Apr 14 04:20:54 user nova-compute[71317]:     </disk>
Apr 14 04:20:54 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:20:54 user nova-compute[71317]:       <mac address="fa:16:3e:5d:8f:83"/>
Apr 14 04:20:54 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:54 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:20:54 user nova-compute[71317]:       <target dev="tap3a3e3b13-7b"/>
Apr 14 04:20:54 user nova-compute[71317]:     </interface>
Apr 14 04:20:54 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:20:54 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/console.log" append="off"/>
Apr 14 04:20:54 user nova-compute[71317]:     </serial>
Apr 14 04:20:54 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:20:54 user nova-compute[71317]:     <video>
Apr 14 04:20:54 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:20:54 user nova-compute[71317]:     </video>
Apr 14 04:20:54 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:20:54 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:20:54 user nova-compute[71317]:     </rng>
Apr 14 04:20:54 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:20:54 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:20:54 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:20:54 user nova-compute[71317]:     </memballoon>
Apr 14 04:20:54 user nova-compute[71317]:   </devices>
Apr 14 04:20:54 user nova-compute[71317]: </domain>
Apr 14 04:20:54 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1350614351',display_name='tempest-ServerActionsTestJSON-server-1398117926',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1350614351',id=6,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:01Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-789bnjb3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:20:46Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=a0a20aeb-ed06-4c65-b87d-74f8f825457d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG os_vif [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Plugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3a3e3b13-7b, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap3a3e3b13-7b, col_values=(('external_ids', {'iface-id': '3a3e3b13-7b4e-46c8-9123-5c1f121a38b9', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:5d:8f:83', 'vm-uuid': 'a0a20aeb-ed06-4c65-b87d-74f8f825457d'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:54 user nova-compute[71317]: INFO os_vif [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully plugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b')
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No VIF found with MAC fa:16:3e:5d:8f:83, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-41e6683a-4399-418a-b065-b5c00e4abc21 req-888556f7-a333-4f89-b1ba-b2053fc82b21 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-41e6683a-4399-418a-b065-b5c00e4abc21 req-888556f7-a333-4f89-b1ba-b2053fc82b21 service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-41e6683a-4399-418a-b065-b5c00e4abc21 req-888556f7-a333-4f89-b1ba-b2053fc82b21 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-41e6683a-4399-418a-b065-b5c00e4abc21 req-888556f7-a333-4f89-b1ba-b2053fc82b21 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-41e6683a-4399-418a-b065-b5c00e4abc21 req-888556f7-a333-4f89-b1ba-b2053fc82b21 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:54 user nova-compute[71317]: WARNING nova.compute.manager [req-41e6683a-4399-418a-b065-b5c00e4abc21 req-888556f7-a333-4f89-b1ba-b2053fc82b21 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state active and task_state rescuing.
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446054.5593045, cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:54 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] VM Resumed (Lifecycle Event)
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5627d40d-113f-45ab-8a0d-f75b213c3136 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:20:54 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446054.574154, cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:54 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] VM Started (Lifecycle Event)
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:20:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "313835fa-6405-41ba-bc89-c4bfc119326a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "313835fa-6405-41ba-bc89-c4bfc119326a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:55 user nova-compute[71317]: INFO nova.compute.manager [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Terminating instance
Apr 14 04:20:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:20:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-b4cc2972-b44e-4201-8d0e-6477f84a0980 req-c3f02e0d-38c2-4758-b9d1-43d5acf5efa4 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b4cc2972-b44e-4201-8d0e-6477f84a0980 req-c3f02e0d-38c2-4758-b9d1-43d5acf5efa4 service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b4cc2972-b44e-4201-8d0e-6477f84a0980 req-c3f02e0d-38c2-4758-b9d1-43d5acf5efa4 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b4cc2972-b44e-4201-8d0e-6477f84a0980 req-c3f02e0d-38c2-4758-b9d1-43d5acf5efa4 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-b4cc2972-b44e-4201-8d0e-6477f84a0980 req-c3f02e0d-38c2-4758-b9d1-43d5acf5efa4 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:56 user nova-compute[71317]: WARNING nova.compute.manager [req-b4cc2972-b44e-4201-8d0e-6477f84a0980 req-c3f02e0d-38c2-4758-b9d1-43d5acf5efa4 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state active and task_state rebuild_spawning.
Apr 14 04:20:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Instance destroyed successfully.
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lazy-loading 'resources' on Instance uuid 313835fa-6405-41ba-bc89-c4bfc119326a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:19:03Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-667634161',display_name='tempest-VolumesActionsTest-instance-667634161',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-667634161',id=15,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:19:10Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='280aa55745a749b7baa134334d25c9c3',ramdisk_id='',reservation_id='r-1cpx0cta',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-447148503',owner_user_name='tempest-VolumesActionsTest-447148503-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:19:11Z,user_data=None,user_id='6e422970cd8340bf996a135088094f51',uuid=313835fa-6405-41ba-bc89-c4bfc119326a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "243334e0-8471-4569-bb6a-38c530471bb4", "address": "fa:16:3e:59:a5:ef", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap243334e0-84", "ovs_interfaceid": "243334e0-8471-4569-bb6a-38c530471bb4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Converting VIF {"id": "243334e0-8471-4569-bb6a-38c530471bb4", "address": "fa:16:3e:59:a5:ef", "network": {"id": "3019ebd6-056f-4b16-98f1-6aabc843f05d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-254689776-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "280aa55745a749b7baa134334d25c9c3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap243334e0-84", "ovs_interfaceid": "243334e0-8471-4569-bb6a-38c530471bb4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:59:a5:ef,bridge_name='br-int',has_traffic_filtering=True,id=243334e0-8471-4569-bb6a-38c530471bb4,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap243334e0-84') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG os_vif [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:59:a5:ef,bridge_name='br-int',has_traffic_filtering=True,id=243334e0-8471-4569-bb6a-38c530471bb4,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap243334e0-84') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-b81ec4a6-5a82-45e7-8c46-5e422002d8c6 req-0462c0bc-7c9b-477a-83d3-34684724cc15 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Received event network-vif-unplugged-243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b81ec4a6-5a82-45e7-8c46-5e422002d8c6 req-0462c0bc-7c9b-477a-83d3-34684724cc15 service nova] Acquiring lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b81ec4a6-5a82-45e7-8c46-5e422002d8c6 req-0462c0bc-7c9b-477a-83d3-34684724cc15 service nova] Lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b81ec4a6-5a82-45e7-8c46-5e422002d8c6 req-0462c0bc-7c9b-477a-83d3-34684724cc15 service nova] Lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-b81ec4a6-5a82-45e7-8c46-5e422002d8c6 req-0462c0bc-7c9b-477a-83d3-34684724cc15 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] No waiting events found dispatching network-vif-unplugged-243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-b81ec4a6-5a82-45e7-8c46-5e422002d8c6 req-0462c0bc-7c9b-477a-83d3-34684724cc15 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Received event network-vif-unplugged-243334e0-8471-4569-bb6a-38c530471bb4 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-b81ec4a6-5a82-45e7-8c46-5e422002d8c6 req-0462c0bc-7c9b-477a-83d3-34684724cc15 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Received event network-vif-plugged-243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b81ec4a6-5a82-45e7-8c46-5e422002d8c6 req-0462c0bc-7c9b-477a-83d3-34684724cc15 service nova] Acquiring lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b81ec4a6-5a82-45e7-8c46-5e422002d8c6 req-0462c0bc-7c9b-477a-83d3-34684724cc15 service nova] Lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b81ec4a6-5a82-45e7-8c46-5e422002d8c6 req-0462c0bc-7c9b-477a-83d3-34684724cc15 service nova] Lock "313835fa-6405-41ba-bc89-c4bfc119326a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-b81ec4a6-5a82-45e7-8c46-5e422002d8c6 req-0462c0bc-7c9b-477a-83d3-34684724cc15 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] No waiting events found dispatching network-vif-plugged-243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:56 user nova-compute[71317]: WARNING nova.compute.manager [req-b81ec4a6-5a82-45e7-8c46-5e422002d8c6 req-0462c0bc-7c9b-477a-83d3-34684724cc15 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Received unexpected event network-vif-plugged-243334e0-8471-4569-bb6a-38c530471bb4 for instance with vm_state active and task_state deleting.
Apr 14 04:20:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap243334e0-84, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:20:56 user nova-compute[71317]: INFO os_vif [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:59:a5:ef,bridge_name='br-int',has_traffic_filtering=True,id=243334e0-8471-4569-bb6a-38c530471bb4,network=Network(3019ebd6-056f-4b16-98f1-6aabc843f05d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap243334e0-84')
Apr 14 04:20:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Deleting instance files /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a_del
Apr 14 04:20:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Deletion of /opt/stack/data/nova/instances/313835fa-6405-41ba-bc89-c4bfc119326a_del complete
Apr 14 04:20:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:56 user nova-compute[71317]: INFO nova.compute.manager [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Took 1.78 seconds to destroy the instance on the hypervisor.
Apr 14 04:20:56 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:20:56 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:20:57 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:20:57 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Took 0.66 seconds to deallocate network for instance.
Apr 14 04:20:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:20:57 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446042.570437, a0a20aeb-ed06-4c65-b87d-74f8f825457d => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:57 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] VM Stopped (Lifecycle Event)
Apr 14 04:20:57 user nova-compute[71317]: DEBUG nova.compute.manager [None req-124aa504-a2e2-46f3-9149-89e63174168f None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:57 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.490s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:58 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Deleted allocations for instance 313835fa-6405-41ba-bc89-c4bfc119326a
Apr 14 04:20:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a95b504b-5425-4206-aef6-da7fd65a3a46 tempest-VolumesActionsTest-447148503 tempest-VolumesActionsTest-447148503-project-member] Lock "313835fa-6405-41ba-bc89-c4bfc119326a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.188s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-10d029aa-3fdf-4650-be5d-fe374c079b4a req-b3d9886a-3ff0-49d0-8270-b8050b71e873 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-10d029aa-3fdf-4650-be5d-fe374c079b4a req-b3d9886a-3ff0-49d0-8270-b8050b71e873 service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-10d029aa-3fdf-4650-be5d-fe374c079b4a req-b3d9886a-3ff0-49d0-8270-b8050b71e873 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-10d029aa-3fdf-4650-be5d-fe374c079b4a req-b3d9886a-3ff0-49d0-8270-b8050b71e873 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-10d029aa-3fdf-4650-be5d-fe374c079b4a req-b3d9886a-3ff0-49d0-8270-b8050b71e873 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:20:58 user nova-compute[71317]: WARNING nova.compute.manager [req-10d029aa-3fdf-4650-be5d-fe374c079b4a req-b3d9886a-3ff0-49d0-8270-b8050b71e873 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state active and task_state rebuild_spawning.
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-10d029aa-3fdf-4650-be5d-fe374c079b4a req-b3d9886a-3ff0-49d0-8270-b8050b71e873 service nova] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Received event network-vif-deleted-243334e0-8471-4569-bb6a-38c530471bb4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446058.8399842, a0a20aeb-ed06-4c65-b87d-74f8f825457d => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:58 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] VM Resumed (Lifecycle Event)
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:20:58 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance spawned successfully.
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rebuild_spawning, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:20:58 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] During sync_power_state the instance has a pending task (rebuild_spawning). Skip.
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446058.840222, a0a20aeb-ed06-4c65-b87d-74f8f825457d => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:20:58 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] VM Started (Lifecycle Event)
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rebuild_spawning, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:20:58 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] During sync_power_state the instance has a pending task (rebuild_spawning). Skip.
Apr 14 04:20:59 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:20:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:20:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:20:59 user nova-compute[71317]: DEBUG nova.objects.instance [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Trying to apply a migration context that does not seem to be set for this instance {{(pid=71317) apply_migration_context /opt/stack/nova/nova/objects/instance.py:1032}}
Apr 14 04:20:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8fa1968a-9ef4-4862-8abd-d310b36852da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: held 0.078s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:11 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446056.5654495, 313835fa-6405-41ba-bc89-c4bfc119326a => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:21:11 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] VM Stopped (Lifecycle Event)
Apr 14 04:21:11 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ca03f481-5a3a-402a-a1c2-e7c3ca02c8e1 None None] [instance: 313835fa-6405-41ba-bc89-c4bfc119326a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:21:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:21:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:21:16 user nova-compute[71317]: INFO nova.compute.claims [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Claim successful on node user
Apr 14 04:21:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:16 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:21:16 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:21:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.386s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:21:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:21:16 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:21:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:21:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG nova.policy [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '6ec7103e06e643a6b9b2c908ba8d7b06', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd5995b934b0546c0983387e46e3b3df4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:21:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Creating image(s)
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "/opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "/opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "/opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk 1073741824" returned: 0 in 0.238s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.374s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Checking if we can resize image /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Successfully created port: 01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Cannot resize image /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'migration_context' on Instance uuid 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Ensure instance console log exists: /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Successfully updated port: 01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:21:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:21:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquired lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:21:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:21:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-1b046d2c-c293-4f4f-8a7a-f3ade7728c75 req-6ef4a5f0-e010-4d3d-be23-1fd213585ede service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Received event network-changed-01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:21:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-1b046d2c-c293-4f4f-8a7a-f3ade7728c75 req-6ef4a5f0-e010-4d3d-be23-1fd213585ede service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Refreshing instance network info cache due to event network-changed-01c99489-4391-41b0-81d9-212d7e76b28c. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:21:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1b046d2c-c293-4f4f-8a7a-f3ade7728c75 req-6ef4a5f0-e010-4d3d-be23-1fd213585ede service nova] Acquiring lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:21:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Updating instance_info_cache with network_info: [{"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Releasing lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Instance network_info: |[{"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1b046d2c-c293-4f4f-8a7a-f3ade7728c75 req-6ef4a5f0-e010-4d3d-be23-1fd213585ede service nova] Acquired lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.network.neutron [req-1b046d2c-c293-4f4f-8a7a-f3ade7728c75 req-6ef4a5f0-e010-4d3d-be23-1fd213585ede service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Refreshing network info cache for port 01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Start _get_guest_xml network_info=[{"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:21:19 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:21:19 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:21:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1631037722',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1631037722',id=18,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-8nem9px3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:21:17Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=226b0e81-b6a2-4094-b0f1-3fb0d3b94074,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:59:f9:6e,bridge_name='br-int',has_traffic_filtering=True,id=01c99489-4391-41b0-81d9-212d7e76b28c,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap01c99489-43') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.objects.instance [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'pci_devices' on Instance uuid 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:21:19 user nova-compute[71317]:   <uuid>226b0e81-b6a2-4094-b0f1-3fb0d3b94074</uuid>
Apr 14 04:21:19 user nova-compute[71317]:   <name>instance-00000012</name>
Apr 14 04:21:19 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:21:19 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:21:19 user nova-compute[71317]:   <metadata>
Apr 14 04:21:19 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:21:19 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:21:19 user nova-compute[71317]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1631037722</nova:name>
Apr 14 04:21:19 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:21:19</nova:creationTime>
Apr 14 04:21:19 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:21:19 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:21:19 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:21:19 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:21:19 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:21:19 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:21:19 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:21:19 user nova-compute[71317]:       <nova:owner>
Apr 14 04:21:19 user nova-compute[71317]:         <nova:user uuid="6ec7103e06e643a6b9b2c908ba8d7b06">tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member</nova:user>
Apr 14 04:21:19 user nova-compute[71317]:         <nova:project uuid="d5995b934b0546c0983387e46e3b3df4">tempest-ServerBootFromVolumeStableRescueTest-247821842</nova:project>
Apr 14 04:21:19 user nova-compute[71317]:       </nova:owner>
Apr 14 04:21:19 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:21:19 user nova-compute[71317]:       <nova:ports>
Apr 14 04:21:19 user nova-compute[71317]:         <nova:port uuid="01c99489-4391-41b0-81d9-212d7e76b28c">
Apr 14 04:21:19 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 04:21:19 user nova-compute[71317]:         </nova:port>
Apr 14 04:21:19 user nova-compute[71317]:       </nova:ports>
Apr 14 04:21:19 user nova-compute[71317]:     </nova:instance>
Apr 14 04:21:19 user nova-compute[71317]:   </metadata>
Apr 14 04:21:19 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:21:19 user nova-compute[71317]:     <system>
Apr 14 04:21:19 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:21:19 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:21:19 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:21:19 user nova-compute[71317]:       <entry name="serial">226b0e81-b6a2-4094-b0f1-3fb0d3b94074</entry>
Apr 14 04:21:19 user nova-compute[71317]:       <entry name="uuid">226b0e81-b6a2-4094-b0f1-3fb0d3b94074</entry>
Apr 14 04:21:19 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:21:19 user nova-compute[71317]:     </system>
Apr 14 04:21:19 user nova-compute[71317]:   </sysinfo>
Apr 14 04:21:19 user nova-compute[71317]:   <os>
Apr 14 04:21:19 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:21:19 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:21:19 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:21:19 user nova-compute[71317]:   </os>
Apr 14 04:21:19 user nova-compute[71317]:   <features>
Apr 14 04:21:19 user nova-compute[71317]:     <acpi/>
Apr 14 04:21:19 user nova-compute[71317]:     <apic/>
Apr 14 04:21:19 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:21:19 user nova-compute[71317]:   </features>
Apr 14 04:21:19 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:21:19 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:21:19 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:21:19 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:21:19 user nova-compute[71317]:   </clock>
Apr 14 04:21:19 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:21:19 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:21:19 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:21:19 user nova-compute[71317]:   </cpu>
Apr 14 04:21:19 user nova-compute[71317]:   <devices>
Apr 14 04:21:19 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:21:19 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:21:19 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk"/>
Apr 14 04:21:19 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:21:19 user nova-compute[71317]:     </disk>
Apr 14 04:21:19 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:21:19 user nova-compute[71317]:       <mac address="fa:16:3e:59:f9:6e"/>
Apr 14 04:21:19 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:21:19 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:21:19 user nova-compute[71317]:       <target dev="tap01c99489-43"/>
Apr 14 04:21:19 user nova-compute[71317]:     </interface>
Apr 14 04:21:19 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:21:19 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/console.log" append="off"/>
Apr 14 04:21:19 user nova-compute[71317]:     </serial>
Apr 14 04:21:19 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:21:19 user nova-compute[71317]:     <video>
Apr 14 04:21:19 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:21:19 user nova-compute[71317]:     </video>
Apr 14 04:21:19 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:21:19 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:21:19 user nova-compute[71317]:     </rng>
Apr 14 04:21:19 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:21:19 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:21:19 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:21:19 user nova-compute[71317]:     </memballoon>
Apr 14 04:21:19 user nova-compute[71317]:   </devices>
Apr 14 04:21:19 user nova-compute[71317]: </domain>
Apr 14 04:21:19 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:21:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1631037722',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1631037722',id=18,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-8nem9px3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:21:17Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=226b0e81-b6a2-4094-b0f1-3fb0d3b94074,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:59:f9:6e,bridge_name='br-int',has_traffic_filtering=True,id=01c99489-4391-41b0-81d9-212d7e76b28c,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap01c99489-43') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG os_vif [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:59:f9:6e,bridge_name='br-int',has_traffic_filtering=True,id=01c99489-4391-41b0-81d9-212d7e76b28c,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap01c99489-43') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap01c99489-43, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap01c99489-43, col_values=(('external_ids', {'iface-id': '01c99489-4391-41b0-81d9-212d7e76b28c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:59:f9:6e', 'vm-uuid': '226b0e81-b6a2-4094-b0f1-3fb0d3b94074'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:19 user nova-compute[71317]: INFO os_vif [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:59:f9:6e,bridge_name='br-int',has_traffic_filtering=True,id=01c99489-4391-41b0-81d9-212d7e76b28c,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap01c99489-43')
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No VIF found with MAC fa:16:3e:59:f9:6e, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.network.neutron [req-1b046d2c-c293-4f4f-8a7a-f3ade7728c75 req-6ef4a5f0-e010-4d3d-be23-1fd213585ede service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Updated VIF entry in instance network info cache for port 01c99489-4391-41b0-81d9-212d7e76b28c. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG nova.network.neutron [req-1b046d2c-c293-4f4f-8a7a-f3ade7728c75 req-6ef4a5f0-e010-4d3d-be23-1fd213585ede service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Updating instance_info_cache with network_info: [{"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:21:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1b046d2c-c293-4f4f-8a7a-f3ade7728c75 req-6ef4a5f0-e010-4d3d-be23-1fd213585ede service nova] Releasing lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:21:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-6a5eaebf-879f-40e9-92d7-0a2f3ef4e4c4 req-b70e4619-2639-49bd-a7b8-776cdbea5aad service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Received event network-vif-plugged-01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:21:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6a5eaebf-879f-40e9-92d7-0a2f3ef4e4c4 req-b70e4619-2639-49bd-a7b8-776cdbea5aad service nova] Acquiring lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6a5eaebf-879f-40e9-92d7-0a2f3ef4e4c4 req-b70e4619-2639-49bd-a7b8-776cdbea5aad service nova] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6a5eaebf-879f-40e9-92d7-0a2f3ef4e4c4 req-b70e4619-2639-49bd-a7b8-776cdbea5aad service nova] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-6a5eaebf-879f-40e9-92d7-0a2f3ef4e4c4 req-b70e4619-2639-49bd-a7b8-776cdbea5aad service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] No waiting events found dispatching network-vif-plugged-01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:21:20 user nova-compute[71317]: WARNING nova.compute.manager [req-6a5eaebf-879f-40e9-92d7-0a2f3ef4e4c4 req-b70e4619-2639-49bd-a7b8-776cdbea5aad service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Received unexpected event network-vif-plugged-01c99489-4391-41b0-81d9-212d7e76b28c for instance with vm_state building and task_state spawning.
Apr 14 04:21:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446082.748744, 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:21:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] VM Resumed (Lifecycle Event)
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:21:22 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Instance spawned successfully.
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:21:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446082.7501655, 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:21:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] VM Started (Lifecycle Event)
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:21:22 user nova-compute[71317]: INFO nova.compute.manager [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Took 5.76 seconds to spawn the instance on the hypervisor.
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:21:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-7355a16d-b013-40de-a9a5-87b5947a6bce req-2755e4a1-9697-4c45-8dad-a2d943926ecb service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Received event network-vif-plugged-01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7355a16d-b013-40de-a9a5-87b5947a6bce req-2755e4a1-9697-4c45-8dad-a2d943926ecb service nova] Acquiring lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7355a16d-b013-40de-a9a5-87b5947a6bce req-2755e4a1-9697-4c45-8dad-a2d943926ecb service nova] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7355a16d-b013-40de-a9a5-87b5947a6bce req-2755e4a1-9697-4c45-8dad-a2d943926ecb service nova] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-7355a16d-b013-40de-a9a5-87b5947a6bce req-2755e4a1-9697-4c45-8dad-a2d943926ecb service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] No waiting events found dispatching network-vif-plugged-01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:21:22 user nova-compute[71317]: WARNING nova.compute.manager [req-7355a16d-b013-40de-a9a5-87b5947a6bce req-2755e4a1-9697-4c45-8dad-a2d943926ecb service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Received unexpected event network-vif-plugged-01c99489-4391-41b0-81d9-212d7e76b28c for instance with vm_state building and task_state spawning.
Apr 14 04:21:22 user nova-compute[71317]: INFO nova.compute.manager [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Took 6.47 seconds to build instance.
Apr 14 04:21:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8966a26b-70e1-4499-af21-5b4430319bd4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.566s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:23 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:21:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:21:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:21:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:21:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:21:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating instance_info_cache with network_info: [{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:21:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:21:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:21:23 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:21:23 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:21:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:21:24 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:21:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:25 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:21:26 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:21:27 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:21:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:28 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:21:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:29 user nova-compute[71317]: INFO nova.compute.manager [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Rescuing
Apr 14 04:21:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:21:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquired lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:21:29 user nova-compute[71317]: DEBUG nova.network.neutron [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:21:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:29 user nova-compute[71317]: DEBUG nova.network.neutron [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Updating instance_info_cache with network_info: [{"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:21:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Releasing lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:21:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.compute.manager [req-797f8a77-db6c-46c4-8321-241d5738c332 req-2a908836-03b8-4d84-b403-361ac9e64172 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-unplugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-797f8a77-db6c-46c4-8321-241d5738c332 req-2a908836-03b8-4d84-b403-361ac9e64172 service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-797f8a77-db6c-46c4-8321-241d5738c332 req-2a908836-03b8-4d84-b403-361ac9e64172 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-797f8a77-db6c-46c4-8321-241d5738c332 req-2a908836-03b8-4d84-b403-361ac9e64172 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.compute.manager [req-797f8a77-db6c-46c4-8321-241d5738c332 req-2a908836-03b8-4d84-b403-361ac9e64172 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-unplugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:21:30 user nova-compute[71317]: WARNING nova.compute.manager [req-797f8a77-db6c-46c4-8321-241d5738c332 req-2a908836-03b8-4d84-b403-361ac9e64172 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-unplugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state active and task_state rescuing.
Apr 14 04:21:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:30 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Instance destroyed successfully.
Apr 14 04:21:30 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Attempting rescue
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=71317) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Instance directory exists: not creating {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 04:21:30 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Creating image(s)
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "/opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "/opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "/opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'trusted_certs' on Instance uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.151s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.rescue {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.rescue" returned: 0 in 0.056s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.217s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'migration_context' on Instance uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Start _get_guest_xml network_info=[{"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "vif_mac": "fa:16:3e:fb:7a:65"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'resources' on Instance uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'numa_topology' on Instance uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:21:30 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:21:30 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'vcpu_model' on Instance uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1529931408',display_name='tempest-ServerRescueNegativeTestJSON-server-1529931408',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1529931408',id=5,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:17:00Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='33320dc2f2c14ecfa5abf6574029c36c',ramdisk_id='',reservation_id='r-wldnu0b2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-508041341',owner_user_name='tempest-ServerRescueNegativeTestJSON-508041341-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:17:01Z,user_data=None,user_id='5234914aec8e443b8a38ac6b38729031',uuid=586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "vif_mac": "fa:16:3e:fb:7a:65"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converting VIF {"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "vif_mac": "fa:16:3e:fb:7a:65"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:fb:7a:65,bridge_name='br-int',has_traffic_filtering=True,id=593a3298-746a-48af-96dc-4c572e0d59c2,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap593a3298-74') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'pci_devices' on Instance uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:21:31 user nova-compute[71317]:   <uuid>586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc</uuid>
Apr 14 04:21:31 user nova-compute[71317]:   <name>instance-00000005</name>
Apr 14 04:21:31 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:21:31 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:21:31 user nova-compute[71317]:   <metadata>
Apr 14 04:21:31 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:21:31 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:21:31 user nova-compute[71317]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-1529931408</nova:name>
Apr 14 04:21:31 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:21:30</nova:creationTime>
Apr 14 04:21:31 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:21:31 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:21:31 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:21:31 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:21:31 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:21:31 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:21:31 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:21:31 user nova-compute[71317]:       <nova:owner>
Apr 14 04:21:31 user nova-compute[71317]:         <nova:user uuid="5234914aec8e443b8a38ac6b38729031">tempest-ServerRescueNegativeTestJSON-508041341-project-member</nova:user>
Apr 14 04:21:31 user nova-compute[71317]:         <nova:project uuid="33320dc2f2c14ecfa5abf6574029c36c">tempest-ServerRescueNegativeTestJSON-508041341</nova:project>
Apr 14 04:21:31 user nova-compute[71317]:       </nova:owner>
Apr 14 04:21:31 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:21:31 user nova-compute[71317]:       <nova:ports>
Apr 14 04:21:31 user nova-compute[71317]:         <nova:port uuid="593a3298-746a-48af-96dc-4c572e0d59c2">
Apr 14 04:21:31 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 04:21:31 user nova-compute[71317]:         </nova:port>
Apr 14 04:21:31 user nova-compute[71317]:       </nova:ports>
Apr 14 04:21:31 user nova-compute[71317]:     </nova:instance>
Apr 14 04:21:31 user nova-compute[71317]:   </metadata>
Apr 14 04:21:31 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:21:31 user nova-compute[71317]:     <system>
Apr 14 04:21:31 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:21:31 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:21:31 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:21:31 user nova-compute[71317]:       <entry name="serial">586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc</entry>
Apr 14 04:21:31 user nova-compute[71317]:       <entry name="uuid">586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc</entry>
Apr 14 04:21:31 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:21:31 user nova-compute[71317]:     </system>
Apr 14 04:21:31 user nova-compute[71317]:   </sysinfo>
Apr 14 04:21:31 user nova-compute[71317]:   <os>
Apr 14 04:21:31 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:21:31 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:21:31 user nova-compute[71317]:   </os>
Apr 14 04:21:31 user nova-compute[71317]:   <features>
Apr 14 04:21:31 user nova-compute[71317]:     <acpi/>
Apr 14 04:21:31 user nova-compute[71317]:     <apic/>
Apr 14 04:21:31 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:21:31 user nova-compute[71317]:   </features>
Apr 14 04:21:31 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:21:31 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:21:31 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:21:31 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:21:31 user nova-compute[71317]:   </clock>
Apr 14 04:21:31 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:21:31 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:21:31 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:21:31 user nova-compute[71317]:   </cpu>
Apr 14 04:21:31 user nova-compute[71317]:   <devices>
Apr 14 04:21:31 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:21:31 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:21:31 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.rescue"/>
Apr 14 04:21:31 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:21:31 user nova-compute[71317]:     </disk>
Apr 14 04:21:31 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:21:31 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:21:31 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk"/>
Apr 14 04:21:31 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:21:31 user nova-compute[71317]:     </disk>
Apr 14 04:21:31 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:21:31 user nova-compute[71317]:       <mac address="fa:16:3e:fb:7a:65"/>
Apr 14 04:21:31 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:21:31 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:21:31 user nova-compute[71317]:       <target dev="tap593a3298-74"/>
Apr 14 04:21:31 user nova-compute[71317]:     </interface>
Apr 14 04:21:31 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:21:31 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/console.log" append="off"/>
Apr 14 04:21:31 user nova-compute[71317]:     </serial>
Apr 14 04:21:31 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:21:31 user nova-compute[71317]:     <video>
Apr 14 04:21:31 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:21:31 user nova-compute[71317]:     </video>
Apr 14 04:21:31 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:21:31 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:21:31 user nova-compute[71317]:     </rng>
Apr 14 04:21:31 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:21:31 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:21:31 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:21:31 user nova-compute[71317]:     </memballoon>
Apr 14 04:21:31 user nova-compute[71317]:   </devices>
Apr 14 04:21:31 user nova-compute[71317]: </domain>
Apr 14 04:21:31 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:21:31 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Instance destroyed successfully.
Apr 14 04:21:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No VIF found with MAC fa:16:3e:fb:7a:65, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:21:32 user nova-compute[71317]: WARNING nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state active and task_state rescuing.
Apr 14 04:21:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:21:32 user nova-compute[71317]: WARNING nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state active and task_state rescuing.
Apr 14 04:21:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:21:32 user nova-compute[71317]: WARNING nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state active and task_state rescuing.
Apr 14 04:21:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-unplugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-unplugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:21:32 user nova-compute[71317]: WARNING nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-unplugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state active and task_state rescuing.
Apr 14 04:21:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:21:32 user nova-compute[71317]: WARNING nova.compute.manager [req-bd21ef97-8f1c-4658-a3d2-f9008f02da55 req-6cabe3a4-aea9-4e21-a2d8-f1aea01912d6 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state active and task_state rescuing.
Apr 14 04:21:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.rescue --force-share --output=json" returned: 0 in 0.250s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.rescue --force-share --output=json" returned: 0 in 0.249s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.203s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG nova.compute.manager [req-20e284c9-bef7-45a3-ba17-f7e0bc92a2c9 req-8eed7909-3b34-4266-a61e-b938811cb541 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-20e284c9-bef7-45a3-ba17-f7e0bc92a2c9 req-8eed7909-3b34-4266-a61e-b938811cb541 service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-20e284c9-bef7-45a3-ba17-f7e0bc92a2c9 req-8eed7909-3b34-4266-a61e-b938811cb541 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-20e284c9-bef7-45a3-ba17-f7e0bc92a2c9 req-8eed7909-3b34-4266-a61e-b938811cb541 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG nova.compute.manager [req-20e284c9-bef7-45a3-ba17-f7e0bc92a2c9 req-8eed7909-3b34-4266-a61e-b938811cb541 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:21:34 user nova-compute[71317]: WARNING nova.compute.manager [req-20e284c9-bef7-45a3-ba17-f7e0bc92a2c9 req-8eed7909-3b34-4266-a61e-b938811cb541 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state active and task_state rescuing.
Apr 14 04:21:34 user nova-compute[71317]: DEBUG nova.compute.manager [req-20e284c9-bef7-45a3-ba17-f7e0bc92a2c9 req-8eed7909-3b34-4266-a61e-b938811cb541 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-20e284c9-bef7-45a3-ba17-f7e0bc92a2c9 req-8eed7909-3b34-4266-a61e-b938811cb541 service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-20e284c9-bef7-45a3-ba17-f7e0bc92a2c9 req-8eed7909-3b34-4266-a61e-b938811cb541 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-20e284c9-bef7-45a3-ba17-f7e0bc92a2c9 req-8eed7909-3b34-4266-a61e-b938811cb541 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG nova.compute.manager [req-20e284c9-bef7-45a3-ba17-f7e0bc92a2c9 req-8eed7909-3b34-4266-a61e-b938811cb541 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:21:34 user nova-compute[71317]: WARNING nova.compute.manager [req-20e284c9-bef7-45a3-ba17-f7e0bc92a2c9 req-8eed7909-3b34-4266-a61e-b938811cb541 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state active and task_state rescuing.
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.198s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdg (vdc) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446095.6806302, 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:21:35 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] VM Resumed (Lifecycle Event)
Apr 14 04:21:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-0f42ce3f-8531-48c3-9db8-5f426e8733c4 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:35 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 14 04:21:35 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446095.6955166, 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:21:35 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] VM Started (Lifecycle Event)
Apr 14 04:21:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:21:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:21:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.493s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdh (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:21:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.rescue --force-share --output=json" returned: 0 in 0.146s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:21:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.rescue --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:21:37 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:21:37 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7363MB free_disk=26.448932647705078GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 9b532a44-20b7-4590-babe-684c49598a62 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance da3b4651-aaa0-430d-85b8-c15de386d08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3d9cd123-3212-4a1d-b7c8-cf0441101048 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance d2e6cf32-080b-47b2-b4db-2b776ba85b38 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1792MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:21:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.549s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:21:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:21:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-803f2656-3789-49f3-abc8-ec2468db167a req-d1510086-96dc-43e3-93cb-0a04bdb9a82c service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-changed-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-803f2656-3789-49f3-abc8-ec2468db167a req-d1510086-96dc-43e3-93cb-0a04bdb9a82c service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Refreshing instance network info cache due to event network-changed-ef7670a9-3421-45de-a1cd-976ddb46f2a1. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:22:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-803f2656-3789-49f3-abc8-ec2468db167a req-d1510086-96dc-43e3-93cb-0a04bdb9a82c service nova] Acquiring lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:22:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-803f2656-3789-49f3-abc8-ec2468db167a req-d1510086-96dc-43e3-93cb-0a04bdb9a82c service nova] Acquired lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:22:01 user nova-compute[71317]: DEBUG nova.network.neutron [req-803f2656-3789-49f3-abc8-ec2468db167a req-d1510086-96dc-43e3-93cb-0a04bdb9a82c service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Refreshing network info cache for port ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:22:01 user nova-compute[71317]: DEBUG nova.network.neutron [req-803f2656-3789-49f3-abc8-ec2468db167a req-d1510086-96dc-43e3-93cb-0a04bdb9a82c service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updated VIF entry in instance network info cache for port ef7670a9-3421-45de-a1cd-976ddb46f2a1. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:22:01 user nova-compute[71317]: DEBUG nova.network.neutron [req-803f2656-3789-49f3-abc8-ec2468db167a req-d1510086-96dc-43e3-93cb-0a04bdb9a82c service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updating instance_info_cache with network_info: [{"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-803f2656-3789-49f3-abc8-ec2468db167a req-d1510086-96dc-43e3-93cb-0a04bdb9a82c service nova] Releasing lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-130152a6-0c80-4184-a796-3a21a605f027 req-d1188bd3-5f2f-43bc-b656-33d86a7b0cfc service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-changed-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-130152a6-0c80-4184-a796-3a21a605f027 req-d1188bd3-5f2f-43bc-b656-33d86a7b0cfc service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Refreshing instance network info cache due to event network-changed-ef7670a9-3421-45de-a1cd-976ddb46f2a1. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-130152a6-0c80-4184-a796-3a21a605f027 req-d1188bd3-5f2f-43bc-b656-33d86a7b0cfc service nova] Acquiring lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-130152a6-0c80-4184-a796-3a21a605f027 req-d1188bd3-5f2f-43bc-b656-33d86a7b0cfc service nova] Acquired lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG nova.network.neutron [req-130152a6-0c80-4184-a796-3a21a605f027 req-d1188bd3-5f2f-43bc-b656-33d86a7b0cfc service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Refreshing network info cache for port ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:02 user nova-compute[71317]: INFO nova.compute.manager [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Detaching volume 70317b0d-2515-45ac-825b-207626822497
Apr 14 04:22:02 user nova-compute[71317]: INFO nova.virt.block_device [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Attempting to driver detach volume 70317b0d-2515-45ac-825b-207626822497 from mountpoint /dev/vdb
Apr 14 04:22:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Attempting to detach device vdb from instance 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:22:02 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:22:02 user nova-compute[71317]:   <source dev="/dev/sdb"/>
Apr 14 04:22:02 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:22:02 user nova-compute[71317]:   <serial>70317b0d-2515-45ac-825b-207626822497</serial>
Apr 14 04:22:02 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 04:22:02 user nova-compute[71317]: </disk>
Apr 14 04:22:02 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:22:02 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Successfully detached device vdb from instance 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e from the persistent domain config.
Apr 14 04:22:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:22:02 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:22:02 user nova-compute[71317]:   <source dev="/dev/sdb"/>
Apr 14 04:22:02 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:22:02 user nova-compute[71317]:   <serial>70317b0d-2515-45ac-825b-207626822497</serial>
Apr 14 04:22:02 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 04:22:02 user nova-compute[71317]: </disk>
Apr 14 04:22:02 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG nova.network.neutron [req-130152a6-0c80-4184-a796-3a21a605f027 req-d1188bd3-5f2f-43bc-b656-33d86a7b0cfc service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updated VIF entry in instance network info cache for port ef7670a9-3421-45de-a1cd-976ddb46f2a1. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG nova.network.neutron [req-130152a6-0c80-4184-a796-3a21a605f027 req-d1188bd3-5f2f-43bc-b656-33d86a7b0cfc service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updating instance_info_cache with network_info: [{"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-130152a6-0c80-4184-a796-3a21a605f027 req-d1188bd3-5f2f-43bc-b656-33d86a7b0cfc service nova] Releasing lock "refresh_cache-4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:22:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446123.6977484, 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:22:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Successfully detached device vdb from instance 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e from the live domain config.
Apr 14 04:22:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HkOfdlri', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:22:03 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:22:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.038s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbcdf26a-21d2-406a-b609-4140fb6cdb46]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:22:03 user nova-compute[71317]: No targets found.
Apr 14 04:22:03 user nova-compute[71317]: iSNS:
Apr 14 04:22:03 user nova-compute[71317]: No targets found.
Apr 14 04:22:03 user nova-compute[71317]: STATIC:
Apr 14 04:22:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5
Apr 14 04:22:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e
Apr 14 04:22:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94
Apr 14 04:22:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3
Apr 14 04:22:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac
Apr 14 04:22:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497
Apr 14 04:22:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:22:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449
Apr 14 04:22:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:03 user nova-compute[71317]: FIRMWARE:
Apr 14 04:22:03 user nova-compute[71317]: No targets found.
Apr 14 04:22:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:22:03 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:22:03 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c86f744-a202-4c14-8400-a0095df07563]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d57349cb-001f-4519-88ef-33ed83d587b7]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449 (non-flash)
Apr 14 04:22:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:03 user nova-compute[71317]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449 (non-flash)
Apr 14 04:22:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84ce5a80d0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497'): ({'sdb'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Removing single pathed devices sdb {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.036s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c01486ea-eed7-4608-ad61-77a4c07497cc]: (4, ('path checker states:\nup                  7\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[40c026e9-347d-45f1-9029-ef671045f599]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Flushing IO for device /dev/sdb {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.034s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2e14bd0-ef8a-4eec-a327-187d4b69e026]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.073s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[67925b56-79fd-43cd-b56c-e389b5ee8341]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[16fe1938-d91c-48a5-a897-3291b255fb27]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --logout" returned: 0 in 0.081s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[afc44ed6-dc39-4426-acb0-d420ffa23fde]: (4, ('Logging out of session [sid: 13, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260]\nLogout of [sid: 13, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 13, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260]
Apr 14 04:22:04 user nova-compute[71317]: Logout of [sid: 13, target: iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497, portal: 172.16.0.220,3260] successful.
Apr 14 04:22:04 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:70317b0d-2515-45ac-825b-207626822497 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.036s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[07d4e1fc-a4dd-4fbd-bc84-2d5551279a38]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.506s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] <== disconnect_volume: return (507ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:22:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:05 user nova-compute[71317]: DEBUG nova.objects.instance [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'flavor' on Instance uuid 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-acc239df-616b-4636-a442-daf1ab9f7363 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.940s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-8d09a653-539e-4ad5-8576-0b1c66adf6b2 req-e8f70905-ac57-43b6-8654-ebde07a95ced service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-changed-42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-8d09a653-539e-4ad5-8576-0b1c66adf6b2 req-e8f70905-ac57-43b6-8654-ebde07a95ced service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing instance network info cache due to event network-changed-42a38c57-4357-4a88-91e3-d76bb38534db. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:22:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8d09a653-539e-4ad5-8576-0b1c66adf6b2 req-e8f70905-ac57-43b6-8654-ebde07a95ced service nova] Acquiring lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:22:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8d09a653-539e-4ad5-8576-0b1c66adf6b2 req-e8f70905-ac57-43b6-8654-ebde07a95ced service nova] Acquired lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:22:06 user nova-compute[71317]: DEBUG nova.network.neutron [req-8d09a653-539e-4ad5-8576-0b1c66adf6b2 req-e8f70905-ac57-43b6-8654-ebde07a95ced service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Refreshing network info cache for port 42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:22:07 user nova-compute[71317]: DEBUG nova.network.neutron [req-8d09a653-539e-4ad5-8576-0b1c66adf6b2 req-e8f70905-ac57-43b6-8654-ebde07a95ced service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updated VIF entry in instance network info cache for port 42a38c57-4357-4a88-91e3-d76bb38534db. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:22:07 user nova-compute[71317]: DEBUG nova.network.neutron [req-8d09a653-539e-4ad5-8576-0b1c66adf6b2 req-e8f70905-ac57-43b6-8654-ebde07a95ced service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating instance_info_cache with network_info: [{"id": "42a38c57-4357-4a88-91e3-d76bb38534db", "address": "fa:16:3e:42:d6:fc", "network": {"id": "75bed293-1d30-421a-9656-15f0ead0650b", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1531253437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.77", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap42a38c57-43", "ovs_interfaceid": "42a38c57-4357-4a88-91e3-d76bb38534db", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "address": "fa:16:3e:ab:44:84", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap186b0001-a3", "ovs_interfaceid": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "address": "fa:16:3e:27:63:d9", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.57", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap70e5d70d-d9", "ovs_interfaceid": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "address": "fa:16:3e:7e:6a:92", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.37", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8bfa317e-39", "ovs_interfaceid": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "address": "fa:16:3e:f7:31:a4", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.117", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap97ba6e98-09", "ovs_interfaceid": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "62770b99-cf54-4132-ad17-d1859ddf8443", "address": "fa:16:3e:b2:3f:e0", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap62770b99-cf", "ovs_interfaceid": "62770b99-cf54-4132-ad17-d1859ddf8443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "address": "fa:16:3e:06:02:5f", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7249969b-00", "ovs_interfaceid": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8d09a653-539e-4ad5-8576-0b1c66adf6b2 req-e8f70905-ac57-43b6-8654-ebde07a95ced service nova] Releasing lock "refresh_cache-3d9cd123-3212-4a1d-b7c8-cf0441101048" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:22:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:08 user nova-compute[71317]: INFO nova.compute.manager [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Terminating instance
Apr 14 04:22:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-34be42b8-72c2-4b28-8050-d203835f0b2f req-913ccc5c-029e-4872-a1bf-f50672935fe6 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-34be42b8-72c2-4b28-8050-d203835f0b2f req-913ccc5c-029e-4872-a1bf-f50672935fe6 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-34be42b8-72c2-4b28-8050-d203835f0b2f req-913ccc5c-029e-4872-a1bf-f50672935fe6 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-34be42b8-72c2-4b28-8050-d203835f0b2f req-913ccc5c-029e-4872-a1bf-f50672935fe6 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-34be42b8-72c2-4b28-8050-d203835f0b2f req-913ccc5c-029e-4872-a1bf-f50672935fe6 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-unplugged-42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-34be42b8-72c2-4b28-8050-d203835f0b2f req-913ccc5c-029e-4872-a1bf-f50672935fe6 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-42a38c57-4357-4a88-91e3-d76bb38534db for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-c730e6f2-b37b-4295-aea1-d35859f9f478 req-d2668f54-053e-4de9-9463-6fde10da76fc service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-186b0001-a3cf-40fa-babf-6b7045fbdb26 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c730e6f2-b37b-4295-aea1-d35859f9f478 req-d2668f54-053e-4de9-9463-6fde10da76fc service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c730e6f2-b37b-4295-aea1-d35859f9f478 req-d2668f54-053e-4de9-9463-6fde10da76fc service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c730e6f2-b37b-4295-aea1-d35859f9f478 req-d2668f54-053e-4de9-9463-6fde10da76fc service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-c730e6f2-b37b-4295-aea1-d35859f9f478 req-d2668f54-053e-4de9-9463-6fde10da76fc service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-unplugged-186b0001-a3cf-40fa-babf-6b7045fbdb26 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-c730e6f2-b37b-4295-aea1-d35859f9f478 req-d2668f54-053e-4de9-9463-6fde10da76fc service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-186b0001-a3cf-40fa-babf-6b7045fbdb26 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:22:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Instance destroyed successfully.
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.objects.instance [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lazy-loading 'resources' on Instance uuid 3d9cd123-3212-4a1d-b7c8-cf0441101048 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:20:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:20:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "42a38c57-4357-4a88-91e3-d76bb38534db", "address": "fa:16:3e:42:d6:fc", "network": {"id": "75bed293-1d30-421a-9656-15f0ead0650b", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1531253437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.77", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap42a38c57-43", "ovs_interfaceid": "42a38c57-4357-4a88-91e3-d76bb38534db", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "42a38c57-4357-4a88-91e3-d76bb38534db", "address": "fa:16:3e:42:d6:fc", "network": {"id": "75bed293-1d30-421a-9656-15f0ead0650b", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1531253437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.77", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap42a38c57-43", "ovs_interfaceid": "42a38c57-4357-4a88-91e3-d76bb38534db", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:42:d6:fc,bridge_name='br-int',has_traffic_filtering=True,id=42a38c57-4357-4a88-91e3-d76bb38534db,network=Network(75bed293-1d30-421a-9656-15f0ead0650b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap42a38c57-43') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:42:d6:fc,bridge_name='br-int',has_traffic_filtering=True,id=42a38c57-4357-4a88-91e3-d76bb38534db,network=Network(75bed293-1d30-421a-9656-15f0ead0650b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap42a38c57-43') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap42a38c57-43, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: INFO os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:42:d6:fc,bridge_name='br-int',has_traffic_filtering=True,id=42a38c57-4357-4a88-91e3-d76bb38534db,network=Network(75bed293-1d30-421a-9656-15f0ead0650b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap42a38c57-43')
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:20:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:20:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "address": "fa:16:3e:ab:44:84", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap186b0001-a3", "ovs_interfaceid": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "address": "fa:16:3e:ab:44:84", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap186b0001-a3", "ovs_interfaceid": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ab:44:84,bridge_name='br-int',has_traffic_filtering=True,id=186b0001-a3cf-40fa-babf-6b7045fbdb26,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap186b0001-a3') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ab:44:84,bridge_name='br-int',has_traffic_filtering=True,id=186b0001-a3cf-40fa-babf-6b7045fbdb26,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap186b0001-a3') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap186b0001-a3, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: INFO os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ab:44:84,bridge_name='br-int',has_traffic_filtering=True,id=186b0001-a3cf-40fa-babf-6b7045fbdb26,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap186b0001-a3')
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:20:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:20:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "address": "fa:16:3e:27:63:d9", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.57", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap70e5d70d-d9", "ovs_interfaceid": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "address": "fa:16:3e:27:63:d9", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.57", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap70e5d70d-d9", "ovs_interfaceid": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:27:63:d9,bridge_name='br-int',has_traffic_filtering=True,id=70e5d70d-d9af-4853-9de8-2cbff2433c60,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap70e5d70d-d9') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:27:63:d9,bridge_name='br-int',has_traffic_filtering=True,id=70e5d70d-d9af-4853-9de8-2cbff2433c60,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap70e5d70d-d9') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap70e5d70d-d9, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: INFO os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:27:63:d9,bridge_name='br-int',has_traffic_filtering=True,id=70e5d70d-d9af-4853-9de8-2cbff2433c60,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap70e5d70d-d9')
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:20:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:20:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "address": "fa:16:3e:7e:6a:92", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.37", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8bfa317e-39", "ovs_interfaceid": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "address": "fa:16:3e:7e:6a:92", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.37", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8bfa317e-39", "ovs_interfaceid": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7e:6a:92,bridge_name='br-int',has_traffic_filtering=True,id=8bfa317e-3995-442d-b5ab-cb1199cb1fb8,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8bfa317e-39') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:7e:6a:92,bridge_name='br-int',has_traffic_filtering=True,id=8bfa317e-3995-442d-b5ab-cb1199cb1fb8,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8bfa317e-39') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8bfa317e-39, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: INFO os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:7e:6a:92,bridge_name='br-int',has_traffic_filtering=True,id=8bfa317e-3995-442d-b5ab-cb1199cb1fb8,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8bfa317e-39')
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:20:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:20:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "address": "fa:16:3e:f7:31:a4", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.117", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap97ba6e98-09", "ovs_interfaceid": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "address": "fa:16:3e:f7:31:a4", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.117", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap97ba6e98-09", "ovs_interfaceid": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=97ba6e98-092c-472f-9f27-eab96fe7f9ae,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap97ba6e98-09') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=97ba6e98-092c-472f-9f27-eab96fe7f9ae,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap97ba6e98-09') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap97ba6e98-09, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: INFO os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=97ba6e98-092c-472f-9f27-eab96fe7f9ae,network=Network(da4ee7a9-dc57-4a23-9799-18b5841681e0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap97ba6e98-09')
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:20:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:20:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "62770b99-cf54-4132-ad17-d1859ddf8443", "address": "fa:16:3e:b2:3f:e0", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap62770b99-cf", "ovs_interfaceid": "62770b99-cf54-4132-ad17-d1859ddf8443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "62770b99-cf54-4132-ad17-d1859ddf8443", "address": "fa:16:3e:b2:3f:e0", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap62770b99-cf", "ovs_interfaceid": "62770b99-cf54-4132-ad17-d1859ddf8443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b2:3f:e0,bridge_name='br-int',has_traffic_filtering=True,id=62770b99-cf54-4132-ad17-d1859ddf8443,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap62770b99-cf') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b2:3f:e0,bridge_name='br-int',has_traffic_filtering=True,id=62770b99-cf54-4132-ad17-d1859ddf8443,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap62770b99-cf') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap62770b99-cf, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-186b0001-a3cf-40fa-babf-6b7045fbdb26 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-186b0001-a3cf-40fa-babf-6b7045fbdb26 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:10 user nova-compute[71317]: WARNING nova.compute.manager [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-186b0001-a3cf-40fa-babf-6b7045fbdb26 for instance with vm_state active and task_state deleting.
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-70e5d70d-d9af-4853-9de8-2cbff2433c60 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-unplugged-70e5d70d-d9af-4853-9de8-2cbff2433c60 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-70e5d70d-d9af-4853-9de8-2cbff2433c60 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-70e5d70d-d9af-4853-9de8-2cbff2433c60 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-70e5d70d-d9af-4853-9de8-2cbff2433c60 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:10 user nova-compute[71317]: WARNING nova.compute.manager [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-70e5d70d-d9af-4853-9de8-2cbff2433c60 for instance with vm_state active and task_state deleting.
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-unplugged-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-9fac2a39-2b3a-4375-b98f-2ec671d44efc req-55a305ea-b869-4a09-b544-5a2798ffe75b service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: INFO os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b2:3f:e0,bridge_name='br-int',has_traffic_filtering=True,id=62770b99-cf54-4132-ad17-d1859ddf8443,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap62770b99-cf')
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T04:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-313960955',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-313960955',id=16,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhfowbNeLBPLrfbqc+PhVghsx2O4QBgG+5Dt4S+mi9KWhCc2XAzzmjd1lHve17iuHYGnqOklxHT1/98oRwn+cDqXux6c/KdzPdG0zM44c8lW6RgvHpOzy6/wjcQqM6ouQ==',key_name='tempest-keypair-489103115',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:20:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3162a049880f4b85afda331443cf52f1',ramdisk_id='',reservation_id='r-zkvtvlmn',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-914882220',owner_user_name='tempest-TaggedBootDevicesTest-914882220-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:20:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3537afcf5a3e47deb53acb34d00e0e83',uuid=3d9cd123-3212-4a1d-b7c8-cf0441101048,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "address": "fa:16:3e:06:02:5f", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7249969b-00", "ovs_interfaceid": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converting VIF {"id": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "address": "fa:16:3e:06:02:5f", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7249969b-00", "ovs_interfaceid": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:06:02:5f,bridge_name='br-int',has_traffic_filtering=True,id=7249969b-00f8-4c4d-8573-832c3e0eb8fe,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7249969b-00') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:06:02:5f,bridge_name='br-int',has_traffic_filtering=True,id=7249969b-00f8-4c4d-8573-832c3e0eb8fe,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7249969b-00') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7249969b-00, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:10 user nova-compute[71317]: INFO os_vif [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:06:02:5f,bridge_name='br-int',has_traffic_filtering=True,id=7249969b-00f8-4c4d-8573-832c3e0eb8fe,network=Network(a12c4e78-0287-4d38-83b1-addfcd1f4315),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7249969b-00')
Apr 14 04:22:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FdyQafHs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:22:10 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:22:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:22:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[26515db6-411e-4dca-ae75-cc391361dfcc]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:22:11 user nova-compute[71317]: No targets found.
Apr 14 04:22:11 user nova-compute[71317]: iSNS:
Apr 14 04:22:11 user nova-compute[71317]: No targets found.
Apr 14 04:22:11 user nova-compute[71317]: STATIC:
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: FIRMWARE:
Apr 14 04:22:11 user nova-compute[71317]: No targets found.
Apr 14 04:22:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:22:11 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:22:11 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcf066e1-cf1c-4332-b2a6-c8fec77868be]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bf1b5986-948e-4505-a2d8-ed9197d31ead]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84b4185a20>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94'): ({'sdd'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Removing single pathed devices sdd {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4463e3d6-4572-49cf-a854-edc7a30ba3bd]: (4, ('path checker states:\nup                  6\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[17cbdffb-cf9d-40bd-853b-13a5aaf7be9f]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Flushing IO for device /dev/sdd {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[83db8ef4-5df1-4ce5-8369-c8d416aa230e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.041s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[85cdb024-4156-424b-a21b-11d581d5958c]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:11 user nova-compute[71317]: WARNING nova.compute.manager [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-42a38c57-4357-4a88-91e3-d76bb38534db for instance with vm_state active and task_state deleting.
Apr 14 04:22:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.004s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-unplugged-97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-97ba6e98-092c-472f-9f27-eab96fe7f9ae for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:11 user nova-compute[71317]: WARNING nova.compute.manager [req-e331d490-3e86-4ac2-b802-a738d8cb251e req-16cd748d-e1eb-43d0-8260-d2221a9aef47 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-97ba6e98-092c-472f-9f27-eab96fe7f9ae for instance with vm_state active and task_state deleting.
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.036s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[67553795-a537-4ca8-93a5-6a4a72224d0c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --logout" returned: 0 in 0.069s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d16a976b-68a1-46d8-9f99-77040fcb2fef]: (4, ('Logging out of session [sid: 15, target: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94, portal: 172.16.0.220,3260]\nLogout of [sid: 15, target: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 15, target: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94, portal: 172.16.0.220,3260]
Apr 14 04:22:11 user nova-compute[71317]: Logout of [sid: 15, target: iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94, portal: 172.16.0.220,3260] successful.
Apr 14 04:22:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f2d01790-bd9f-4eff-aa4d-2248429aeb94 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.036s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b26d54ea-8c60-4878-8a96-15a3fb7b888f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.389s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] <== disconnect_volume: return (390ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'aEOFgACW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:22:11 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b1215cb-5aef-4482-96e6-26c857881790]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:22:11 user nova-compute[71317]: No targets found.
Apr 14 04:22:11 user nova-compute[71317]: iSNS:
Apr 14 04:22:11 user nova-compute[71317]: No targets found.
Apr 14 04:22:11 user nova-compute[71317]: STATIC:
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: FIRMWARE:
Apr 14 04:22:11 user nova-compute[71317]: No targets found.
Apr 14 04:22:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:22:11 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:22:11 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.037s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[62bc9950-73ac-41c2-83f2-4824c8f66b96]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.079s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0364092-ca3f-4d4a-af03-eb6b2c8c0210]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c41e4af0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac'): ({'sde'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Removing single pathed devices sde {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[623b5877-46e9-4854-bbf4-dba0cd56f1b2]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3713ed5-8af6-4d8f-b403-ae0a332b1f6d]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Flushing IO for device /dev/sde {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4d257dd-c06d-4637-992d-f898a1679268]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.061s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca199d62-9851-4fa1-ab9e-85e424cdc873]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[763fc669-1ad6-4f1c-8d2c-4b2aa598a2bf]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --logout" returned: 0 in 0.086s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[672b12f7-b679-481c-9dfa-b2c77b5e5273]: (4, ('Logging out of session [sid: 16, target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac, portal: 172.16.0.220,3260]\nLogout of [sid: 16, target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 16, target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac, portal: 172.16.0.220,3260]
Apr 14 04:22:11 user nova-compute[71317]: Logout of [sid: 16, target: iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac, portal: 172.16.0.220,3260] successful.
Apr 14 04:22:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40e8dfdd-c6e4-4f7f-a465-913fe79829ac -p 172.16.0.220:3260 --op delete" returned: 0 in 0.036s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c8c17bb-a5d3-4b02-a41e-5f4693acca2c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.468s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] <== disconnect_volume: return (469ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'WspLiGZN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:22:11 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[67a49102-60cb-4edb-9216-631c4dbe6e71]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:22:11 user nova-compute[71317]: No targets found.
Apr 14 04:22:11 user nova-compute[71317]: iSNS:
Apr 14 04:22:11 user nova-compute[71317]: No targets found.
Apr 14 04:22:11 user nova-compute[71317]: STATIC:
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449
Apr 14 04:22:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:11 user nova-compute[71317]: FIRMWARE:
Apr 14 04:22:11 user nova-compute[71317]: No targets found.
Apr 14 04:22:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:22:11 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:22:11 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2c9f18d-23fd-46cf-99df-0800a76bb505]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[458440ed-7283-4312-951a-4ea05c3d33a0]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:11 user nova-compute[71317]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:16203803-41c1-4da7-804c-d344751cb449 (non-flash)
Apr 14 04:22:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c41e4af0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3'): ({'sdg'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Removing single pathed devices sdg {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ddf400dc-aeed-420f-976e-92c2a3f317e7]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0e044cd-312d-4527-b635-c506e5e6fe8c]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Flushing IO for device /dev/sdg {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:22:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e81f554-9592-4b89-94c7-57ea5259e452]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.043s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b56e3d8-0558-4f2b-8aee-5b5c1da49f77]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] SCSI volumes sdg have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.037s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[77084773-515f-4649-9db4-68bff9391633]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d125250a-e1d5-4b53-8b36-e296009fcc7b]: (4, ('Logging out of session [sid: 18, target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3, portal: 172.16.0.220,3260]\nLogout of [sid: 18, target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 18, target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3, portal: 172.16.0.220,3260]
Apr 14 04:22:12 user nova-compute[71317]: Logout of [sid: 18, target: iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3, portal: 172.16.0.220,3260] successful.
Apr 14 04:22:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5e5988ce-8d0a-49a0-acc8-84f7d3917bd3 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a54ddb12-e643-431e-bbef-3e944ed70a2d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.347s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] <== disconnect_volume: return (348ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:22:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Deleting instance files /opt/stack/data/nova/instances/3d9cd123-3212-4a1d-b7c8-cf0441101048_del
Apr 14 04:22:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Deletion of /opt/stack/data/nova/instances/3d9cd123-3212-4a1d-b7c8-cf0441101048_del complete
Apr 14 04:22:12 user nova-compute[71317]: INFO nova.compute.manager [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Took 4.11 seconds to destroy the instance on the hypervisor.
Apr 14 04:22:12 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:22:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-1c331273-ae67-4e0e-b411-58340282a44e req-29b5243c-025f-4aae-be69-afaa8da21aea service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1c331273-ae67-4e0e-b411-58340282a44e req-29b5243c-025f-4aae-be69-afaa8da21aea service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1c331273-ae67-4e0e-b411-58340282a44e req-29b5243c-025f-4aae-be69-afaa8da21aea service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1c331273-ae67-4e0e-b411-58340282a44e req-29b5243c-025f-4aae-be69-afaa8da21aea service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-1c331273-ae67-4e0e-b411-58340282a44e req-29b5243c-025f-4aae-be69-afaa8da21aea service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:13 user nova-compute[71317]: WARNING nova.compute.manager [req-1c331273-ae67-4e0e-b411-58340282a44e req-29b5243c-025f-4aae-be69-afaa8da21aea service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 for instance with vm_state active and task_state deleting.
Apr 14 04:22:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-unplugged-62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-62770b99-cf54-4132-ad17-d1859ddf8443 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:13 user nova-compute[71317]: WARNING nova.compute.manager [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-62770b99-cf54-4132-ad17-d1859ddf8443 for instance with vm_state active and task_state deleting.
Apr 14 04:22:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-unplugged-7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-unplugged-7249969b-00f8-4c4d-8573-832c3e0eb8fe for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-plugged-7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] Acquiring lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] No waiting events found dispatching network-vif-plugged-7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:13 user nova-compute[71317]: WARNING nova.compute.manager [req-5df6e6d8-4df7-49a9-a775-ddfa9f96a4fe req-3430002b-f4fb-4497-875c-39f14ba43498 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received unexpected event network-vif-plugged-7249969b-00f8-4c4d-8573-832c3e0eb8fe for instance with vm_state active and task_state deleting.
Apr 14 04:22:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-4da21b9a-8cfb-4512-a657-e95e476458bb req-6d774034-0c9e-4555-bd8b-31e26b0d4862 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-deleted-62770b99-cf54-4132-ad17-d1859ddf8443 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:15 user nova-compute[71317]: INFO nova.compute.manager [req-4da21b9a-8cfb-4512-a657-e95e476458bb req-6d774034-0c9e-4555-bd8b-31e26b0d4862 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Neutron deleted interface 62770b99-cf54-4132-ad17-d1859ddf8443; detaching it from the instance and deleting it from the info cache
Apr 14 04:22:15 user nova-compute[71317]: DEBUG nova.network.neutron [req-4da21b9a-8cfb-4512-a657-e95e476458bb req-6d774034-0c9e-4555-bd8b-31e26b0d4862 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating instance_info_cache with network_info: [{"id": "42a38c57-4357-4a88-91e3-d76bb38534db", "address": "fa:16:3e:42:d6:fc", "network": {"id": "75bed293-1d30-421a-9656-15f0ead0650b", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1531253437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.77", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap42a38c57-43", "ovs_interfaceid": "42a38c57-4357-4a88-91e3-d76bb38534db", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "address": "fa:16:3e:ab:44:84", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap186b0001-a3", "ovs_interfaceid": "186b0001-a3cf-40fa-babf-6b7045fbdb26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "address": "fa:16:3e:27:63:d9", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.57", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap70e5d70d-d9", "ovs_interfaceid": "70e5d70d-d9af-4853-9de8-2cbff2433c60", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "address": "fa:16:3e:7e:6a:92", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.37", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8bfa317e-39", "ovs_interfaceid": "8bfa317e-3995-442d-b5ab-cb1199cb1fb8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "address": "fa:16:3e:f7:31:a4", "network": {"id": "da4ee7a9-dc57-4a23-9799-18b5841681e0", "bridge": "br-int", "label": "tempest-device-tagging-net1-224133014", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.117", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap97ba6e98-09", "ovs_interfaceid": "97ba6e98-092c-472f-9f27-eab96fe7f9ae", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "address": "fa:16:3e:06:02:5f", "network": {"id": "a12c4e78-0287-4d38-83b1-addfcd1f4315", "bridge": "br-int", "label": "tempest-device-tagging-net2-1173917578", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "3162a049880f4b85afda331443cf52f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7249969b-00", "ovs_interfaceid": "7249969b-00f8-4c4d-8573-832c3e0eb8fe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-4da21b9a-8cfb-4512-a657-e95e476458bb req-6d774034-0c9e-4555-bd8b-31e26b0d4862 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Detach interface failed, port_id=62770b99-cf54-4132-ad17-d1859ddf8443, reason: Instance 3d9cd123-3212-4a1d-b7c8-cf0441101048 could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:22:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:16 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:16 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Took 4.38 seconds to deallocate network for instance.
Apr 14 04:22:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-0808a576-5048-476e-94cc-e2b986210609 req-1510ce6a-3472-4a3a-a03f-43f38544cfd5 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-deleted-42a38c57-4357-4a88-91e3-d76bb38534db {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-0808a576-5048-476e-94cc-e2b986210609 req-1510ce6a-3472-4a3a-a03f-43f38544cfd5 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-deleted-7249969b-00f8-4c4d-8573-832c3e0eb8fe {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-0808a576-5048-476e-94cc-e2b986210609 req-1510ce6a-3472-4a3a-a03f-43f38544cfd5 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-deleted-97ba6e98-092c-472f-9f27-eab96fe7f9ae {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-0808a576-5048-476e-94cc-e2b986210609 req-1510ce6a-3472-4a3a-a03f-43f38544cfd5 service nova] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Received event network-vif-deleted-8bfa317e-3995-442d-b5ab-cb1199cb1fb8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:18 user nova-compute[71317]: INFO nova.compute.manager [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Detaching volume b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:22:18 user nova-compute[71317]: INFO nova.virt.block_device [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Attempting to driver detach volume b7e94002-3eeb-442d-b224-ef83dacfa5a4 from mountpoint /dev/vdb
Apr 14 04:22:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Attempting to detach device vdb from instance da3b4651-aaa0-430d-85b8-c15de386d08f from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:22:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:22:18 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:22:18 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:22:18 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:22:18 user nova-compute[71317]:   <serial>b7e94002-3eeb-442d-b224-ef83dacfa5a4</serial>
Apr 14 04:22:18 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 04:22:18 user nova-compute[71317]: </disk>
Apr 14 04:22:18 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:22:18 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully detached device vdb from instance da3b4651-aaa0-430d-85b8-c15de386d08f from the persistent domain config.
Apr 14 04:22:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance da3b4651-aaa0-430d-85b8-c15de386d08f from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:22:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:22:18 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:22:18 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:22:18 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:22:18 user nova-compute[71317]:   <serial>b7e94002-3eeb-442d-b224-ef83dacfa5a4</serial>
Apr 14 04:22:18 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 04:22:18 user nova-compute[71317]: </disk>
Apr 14 04:22:18 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446139.3548656, da3b4651-aaa0-430d-85b8-c15de386d08f => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:22:19 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully detached device vdb from instance da3b4651-aaa0-430d-85b8-c15de386d08f from the live domain config.
Apr 14 04:22:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mfOroQgL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:22:19 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4726a4b4-ed77-492f-ae20-9475ba8885a3]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:22:19 user nova-compute[71317]: No targets found.
Apr 14 04:22:19 user nova-compute[71317]: iSNS:
Apr 14 04:22:19 user nova-compute[71317]: No targets found.
Apr 14 04:22:19 user nova-compute[71317]: STATIC:
Apr 14 04:22:19 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5
Apr 14 04:22:19 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:19 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:19 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e
Apr 14 04:22:19 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:19 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:19 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4
Apr 14 04:22:19 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:19 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:19 user nova-compute[71317]: FIRMWARE:
Apr 14 04:22:19 user nova-compute[71317]: No targets found.
Apr 14 04:22:19 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:22:19 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:22:19 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba8773f7-5bc4-44ca-ae2e-62a1dd0e7c86]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4f1cf26-5652-4271-8c38-435dd95192c2]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:22:19 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:19 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:19 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 (non-flash)
Apr 14 04:22:19 user nova-compute[71317]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:19 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:19 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c4399900>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[56bb8964-8ef3-44fb-a784-a87a092fd7a9]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[42be53f0-e50c-4929-a897-bab35391eff3]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a58984f6-fa27-4938-8819-3dc4cde5e46d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[96bb2a75-2ee6-4d5d-8b3b-40905224ad44]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d573a40-c366-4c51-a880-b4201d22fc20]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --logout" returned: 0 in 0.069s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b680d2b8-c91c-4dcf-82cf-dd4bd70198a4]: (4, ('Logging out of session [sid: 14, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260]\nLogout of [sid: 14, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 14, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260]
Apr 14 04:22:19 user nova-compute[71317]: Logout of [sid: 14, target: iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4, portal: 172.16.0.220,3260] successful.
Apr 14 04:22:19 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b7e94002-3eeb-442d-b224-ef83dacfa5a4 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4295d062-ba0c-439e-96f7-34e326d20506]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.335s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] <== disconnect_volume: return (337ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:22:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:22:20 user nova-compute[71317]: DEBUG nova.objects.instance [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-35eed0d4-ee97-4054-a296-2c72f10a8926 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.923s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7baca928-383a-478b-a594-3ee636f35a78 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7baca928-383a-478b-a594-3ee636f35a78 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" acquired by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7baca928-383a-478b-a594-3ee636f35a78 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7baca928-383a-478b-a594-3ee636f35a78 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Stopping instance; current vm_state: active, current task_state: powering-off, current DB power_state: 1, current VM power_state: 1 {{(pid=71317) do_stop_instance /opt/stack/nova/nova/compute/manager.py:3323}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG nova.objects.instance [None req-7baca928-383a-478b-a594-3ee636f35a78 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-0b1472af-0947-4ba2-ab55-9e3430f0050b req-f321c424-12f0-493a-b1d1-0e03739be693 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-unplugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0b1472af-0947-4ba2-ab55-9e3430f0050b req-f321c424-12f0-493a-b1d1-0e03739be693 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0b1472af-0947-4ba2-ab55-9e3430f0050b req-f321c424-12f0-493a-b1d1-0e03739be693 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0b1472af-0947-4ba2-ab55-9e3430f0050b req-f321c424-12f0-493a-b1d1-0e03739be693 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-0b1472af-0947-4ba2-ab55-9e3430f0050b req-f321c424-12f0-493a-b1d1-0e03739be693 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-unplugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:21 user nova-compute[71317]: WARNING nova.compute.manager [req-0b1472af-0947-4ba2-ab55-9e3430f0050b req-f321c424-12f0-493a-b1d1-0e03739be693 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-unplugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state active and task_state powering-off.
Apr 14 04:22:21 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance destroyed successfully.
Apr 14 04:22:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7baca928-383a-478b-a594-3ee636f35a78 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:22:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7baca928-383a-478b-a594-3ee636f35a78 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" "released" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: held 0.887s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:22 user nova-compute[71317]: INFO nova.compute.manager [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Took 5.55 seconds to detach 3 volumes for instance.
Apr 14 04:22:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:22 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:22:22 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:22:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.391s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:22 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Deleted allocations for instance 3d9cd123-3212-4a1d-b7c8-cf0441101048
Apr 14 04:22:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-64d3668d-ef08-4702-ac04-8fcba8c89fa6 tempest-TaggedBootDevicesTest-914882220 tempest-TaggedBootDevicesTest-914882220-project-member] Lock "3d9cd123-3212-4a1d-b7c8-cf0441101048" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 14.697s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-0a747dd7-e004-4cef-93c6-630bff31b59b req-7ea228ed-2655-4f80-84b2-bcdc89ea557c service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Received event network-changed-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-0a747dd7-e004-4cef-93c6-630bff31b59b req-7ea228ed-2655-4f80-84b2-bcdc89ea557c service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Refreshing instance network info cache due to event network-changed-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0a747dd7-e004-4cef-93c6-630bff31b59b req-7ea228ed-2655-4f80-84b2-bcdc89ea557c service nova] Acquiring lock "refresh_cache-d2e6cf32-080b-47b2-b4db-2b776ba85b38" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0a747dd7-e004-4cef-93c6-630bff31b59b req-7ea228ed-2655-4f80-84b2-bcdc89ea557c service nova] Acquired lock "refresh_cache-d2e6cf32-080b-47b2-b4db-2b776ba85b38" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG nova.network.neutron [req-0a747dd7-e004-4cef-93c6-630bff31b59b req-7ea228ed-2655-4f80-84b2-bcdc89ea557c service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Refreshing network info cache for port 7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-7469f5b6-df9e-41cd-9b55-6c589b4e6f1e req-8bdeddd5-8b2d-4c23-b414-c9a45ba137de service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7469f5b6-df9e-41cd-9b55-6c589b4e6f1e req-8bdeddd5-8b2d-4c23-b414-c9a45ba137de service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7469f5b6-df9e-41cd-9b55-6c589b4e6f1e req-8bdeddd5-8b2d-4c23-b414-c9a45ba137de service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7469f5b6-df9e-41cd-9b55-6c589b4e6f1e req-8bdeddd5-8b2d-4c23-b414-c9a45ba137de service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-7469f5b6-df9e-41cd-9b55-6c589b4e6f1e req-8bdeddd5-8b2d-4c23-b414-c9a45ba137de service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:23 user nova-compute[71317]: WARNING nova.compute.manager [req-7469f5b6-df9e-41cd-9b55-6c589b4e6f1e req-8bdeddd5-8b2d-4c23-b414-c9a45ba137de service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state stopped and task_state None.
Apr 14 04:22:23 user nova-compute[71317]: DEBUG nova.network.neutron [req-0a747dd7-e004-4cef-93c6-630bff31b59b req-7ea228ed-2655-4f80-84b2-bcdc89ea557c service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Updated VIF entry in instance network info cache for port 7c8689e1-9c0a-42b6-8cb3-7751f619ba1f. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG nova.network.neutron [req-0a747dd7-e004-4cef-93c6-630bff31b59b req-7ea228ed-2655-4f80-84b2-bcdc89ea557c service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Updating instance_info_cache with network_info: [{"id": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "address": "fa:16:3e:ae:9a:66", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7c8689e1-9c", "ovs_interfaceid": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0a747dd7-e004-4cef-93c6-630bff31b59b req-7ea228ed-2655-4f80-84b2-bcdc89ea557c service nova] Releasing lock "refresh_cache-d2e6cf32-080b-47b2-b4db-2b776ba85b38" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:22:24 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Updating instance_info_cache with network_info: [{"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:22:24 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:22:24 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:24 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:24 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:24 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:22:25 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:25 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446130.5178807, 3d9cd123-3212-4a1d-b7c8-cf0441101048 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:22:25 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] VM Stopped (Lifecycle Event)
Apr 14 04:22:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-df182c07-09be-4e53-85b6-2f193aed7039 None None] [instance: 3d9cd123-3212-4a1d-b7c8-cf0441101048] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:22:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:26 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:26 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 04:22:26 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] There are 0 instances to clean {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 04:22:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:26 user nova-compute[71317]: DEBUG nova.objects.instance [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lazy-loading 'flavor' on Instance uuid d2e6cf32-080b-47b2-b4db-2b776ba85b38 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.065s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:27 user nova-compute[71317]: INFO nova.compute.manager [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Attaching volume 76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 to /dev/vdb
Apr 14 04:22:27 user nova-compute[71317]: DEBUG os_brick.utils [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9030d376-2f2f-4362-9fdf-131bf08b545b]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[83b2ab5e-7812-4e2a-8bd2-cf155db15512]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9872b76f-20da-4dd5-b378-79a4ff781f86]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[62f2f7d5-fbf1-452e-9252-31ebec1ece79]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:22:27 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:22:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[983968b2-b606-4508-8cff-ccf014744523]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG os_brick.utils [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] <== get_connector_properties: return (137ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Updating existing volume attachment record: 20019e69-02d4-4f64-91b2-d7792a3f9cf3 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:22:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:28 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:28 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.146s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.166s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.rescue --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk.rescue --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.206s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.273s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdh (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:22:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.rescue --force-share --output=json" returned: 0 in 0.179s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk.rescue --force-share --output=json" returned: 0 in 0.167s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:35 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:22:35 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7661MB free_disk=26.3955078125GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 9b532a44-20b7-4590-babe-684c49598a62 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance da3b4651-aaa0-430d-85b8-c15de386d08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance d2e6cf32-080b-47b2-b4db-2b776ba85b38 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing inventories for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating ProviderTree inventory for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating inventory in ProviderTree for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing aggregate associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, aggregates: None {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 04:22:35 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing trait associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, traits: COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSE41,HW_CPU_X86_SSE42,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_TRUSTED_CERTS,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_ACCELERATORS,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NET_VIF_MODEL_E1000,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_NODE,HW_CPU_X86_SSE2,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,HW_CPU_X86_SSSE3,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_STORAGE_BUS_USB,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_NE2K_PCI {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kSWhTJji', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:22:36 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:22:36 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:22:36 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.829s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances with incomplete migration  {{(pid=71317) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260" returned: 21 in 0.035s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[34ddf96a-0d23-4533-85b4-e3a71a406a27]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:22:36 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e6a0cd6-3e12-4374-bb08-daa2b2df26a1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66] added
Apr 14 04:22:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[73aadbf1-86b7-474f-a0d6-383b8628f42e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f219176-c959-49d3-a30d-5c847c4f1494]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kSWhTJji {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kSWhTJji" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ba586c8-bc75-40f0-a7c1-a7e9e8aa0e5c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'kSWhTJji'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e1f4461-00f7-45bd-a88b-896ec91da348]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b22fd00-8853-4560-9cb0-cc7f960e5073]: (4, ('tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:36 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsi session list stdout=tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:36 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --login" returned: 0 in 0.067s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[612e5a3c-18ad-43bf-bbf9-67615be1c9d2]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66, portal: 172.16.0.220,3260]
Apr 14 04:22:36 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66, portal: 172.16.0.220,3260] successful.
Apr 14 04:22:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[04578f1a-4021-4ce9-91cd-685c286eacd0]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[15a5d447-f7e6-4b6f-864f-e54b7752f4ac]: (4, ('tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:36 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:36 user nova-compute[71317]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 (non-flash)
Apr 14 04:22:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsi session list stdout=tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:36 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:36 user nova-compute[71317]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 (non-flash)
Apr 14 04:22:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.326s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] HCTL ('33', '-', '-', 0) found on session 22 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[038d34e0-9038-41fa-b7b7-4ba11d05ca2a]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Searching for a device in session 22 and hctl ['33', '*', '*', 0] yield: None {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446141.8826666, da3b4651-aaa0-430d-85b8-c15de386d08f => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:22:36 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] VM Stopped (Lifecycle Event)
Apr 14 04:22:36 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a5151c61-b4d1-40a0-b454-0d63f6e51eab None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:22:36 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a5151c61-b4d1-40a0-b454-0d63f6e51eab None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: stopped, current task_state: None, current DB power_state: 4, VM power_state: 4 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:22:37 user nova-compute[71317]: INFO nova.compute.manager [None req-342ecc7a-d4c1-4ea1-bc76-9432521f482c tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Unrescuing
Apr 14 04:22:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-342ecc7a-d4c1-4ea1-bc76-9432521f482c tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-342ecc7a-d4c1-4ea1-bc76-9432521f482c tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquired lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG nova.network.neutron [None req-342ecc7a-d4c1-4ea1-bc76-9432521f482c tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Searching for a device in session 22 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kSWhTJji', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.357s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] <== connect_volume: return (1359ms) {'type': 'block', 'scsi_wwn': '23039666264653836', 'path': '/dev/sdb'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23039666264653836', 'path': '/dev/sdb'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG nova.objects.instance [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lazy-loading 'flavor' on Instance uuid d2e6cf32-080b-47b2-b4db-2b776ba85b38 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:22:37 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:22:37 user nova-compute[71317]:   <source dev="/dev/sdb"/>
Apr 14 04:22:37 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:22:37 user nova-compute[71317]:   <serial>76b8598c-a7c5-4cc7-bd79-eb7d98b3da66</serial>
Apr 14 04:22:37 user nova-compute[71317]: </disk>
Apr 14 04:22:37 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG nova.network.neutron [None req-342ecc7a-d4c1-4ea1-bc76-9432521f482c tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updating instance_info_cache with network_info: [{"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-342ecc7a-d4c1-4ea1-bc76-9432521f482c tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Releasing lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG nova.objects.instance [None req-342ecc7a-d4c1-4ea1-bc76-9432521f482c tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'flavor' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:22:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] No VIF found with MAC fa:16:3e:ae:9a:66, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:22:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-f634c40f-d9b6-4518-a7bb-0591e63dfec9 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 11.123s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-215a4420-97cf-4480-b421-55a94514adda req-ad76c381-0a2f-4240-9162-3271f806795e service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-unplugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-215a4420-97cf-4480-b421-55a94514adda req-ad76c381-0a2f-4240-9162-3271f806795e service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-215a4420-97cf-4480-b421-55a94514adda req-ad76c381-0a2f-4240-9162-3271f806795e service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-215a4420-97cf-4480-b421-55a94514adda req-ad76c381-0a2f-4240-9162-3271f806795e service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-215a4420-97cf-4480-b421-55a94514adda req-ad76c381-0a2f-4240-9162-3271f806795e service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-unplugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:38 user nova-compute[71317]: WARNING nova.compute.manager [req-215a4420-97cf-4480-b421-55a94514adda req-ad76c381-0a2f-4240-9162-3271f806795e service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-unplugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:22:38 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Instance destroyed successfully.
Apr 14 04:22:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:39 user nova-compute[71317]: INFO nova.compute.manager [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Detaching volume 76b8598c-a7c5-4cc7-bd79-eb7d98b3da66
Apr 14 04:22:39 user nova-compute[71317]: INFO nova.virt.block_device [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Attempting to driver detach volume 76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 from mountpoint /dev/vdb
Apr 14 04:22:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Attempting to detach device vdb from instance d2e6cf32-080b-47b2-b4db-2b776ba85b38 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:22:39 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:22:39 user nova-compute[71317]:   <source dev="/dev/sdb"/>
Apr 14 04:22:39 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:22:39 user nova-compute[71317]:   <serial>76b8598c-a7c5-4cc7-bd79-eb7d98b3da66</serial>
Apr 14 04:22:39 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:22:39 user nova-compute[71317]: </disk>
Apr 14 04:22:39 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:22:39 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Successfully detached device vdb from instance d2e6cf32-080b-47b2-b4db-2b776ba85b38 from the persistent domain config.
Apr 14 04:22:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance d2e6cf32-080b-47b2-b4db-2b776ba85b38 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:22:39 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:22:39 user nova-compute[71317]:   <source dev="/dev/sdb"/>
Apr 14 04:22:39 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:22:39 user nova-compute[71317]:   <serial>76b8598c-a7c5-4cc7-bd79-eb7d98b3da66</serial>
Apr 14 04:22:39 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:22:39 user nova-compute[71317]: </disk>
Apr 14 04:22:39 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "97392a47-53f0-4bde-977a-e2945332fe1f" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "97392a47-53f0-4bde-977a-e2945332fe1f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:22:39 user nova-compute[71317]: INFO nova.compute.claims [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Claim successful on node user
Apr 14 04:22:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446159.922038, d2e6cf32-080b-47b2-b4db-2b776ba85b38 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance d2e6cf32-080b-47b2-b4db-2b776ba85b38 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:22:39 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Successfully detached device vdb from instance d2e6cf32-080b-47b2-b4db-2b776ba85b38 from the live domain config.
Apr 14 04:22:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kSWhTJji', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:22:39 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:22:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e924d7e-93b3-4d02-a868-619b1b4e5f5f]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:22:39 user nova-compute[71317]: No targets found.
Apr 14 04:22:39 user nova-compute[71317]: iSNS:
Apr 14 04:22:39 user nova-compute[71317]: No targets found.
Apr 14 04:22:39 user nova-compute[71317]: STATIC:
Apr 14 04:22:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5
Apr 14 04:22:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66
Apr 14 04:22:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e
Apr 14 04:22:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:22:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:22:39 user nova-compute[71317]: FIRMWARE:
Apr 14 04:22:39 user nova-compute[71317]: No targets found.
Apr 14 04:22:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:22:39 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:22:39 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.531s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b232877e-c5b2-4bc6-97c6-58b072418b52]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b3afb95-f606-482d-9313-eb6a45d71d72]: (4, ('tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:40 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:40 user nova-compute[71317]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 (non-flash)
Apr 14 04:22:40 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsi session list stdout=tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:22:40 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:22:40 user nova-compute[71317]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 (non-flash)
Apr 14 04:22:40 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8490533a30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66'): ({'sdb'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Removing single pathed devices sdb {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.network.neutron [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[932e0042-5012-41d3-b33e-f3f98c313262]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[505795d9-ff85-45a9-8cf3-a5ed2280f3a3]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Flushing IO for device /dev/sdb {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[105bef30-4b9e-4dea-bbe2-68a50309173d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.033s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d80343ed-75da-4913-8bd7-c66c90802a1a]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e77b750f-4091-491c-8cf9-2402291071bf]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.policy [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e4a70281d6734c15b3c4b1b00e4b5d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0848c9a67e2746e5b12b73920215b729', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:22:40 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Creating image(s)
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "/opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "/opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "/opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --logout" returned: 0 in 0.086s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[64558d94-b5eb-4e22-b173-4d5f620f54bb]: (4, ('Logging out of session [sid: 22, target: iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66, portal: 172.16.0.220,3260]\nLogout of [sid: 22, target: iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 22, target: iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66, portal: 172.16.0.220,3260]
Apr 14 04:22:40 user nova-compute[71317]: Logout of [sid: 22, target: iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66, portal: 172.16.0.220,3260] successful.
Apr 14 04:22:40 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:76b8598c-a7c5-4cc7-bd79-eb7d98b3da66 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c088db48-3ca6-4e87-9211-78e7bdd41027]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.352s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] <== disconnect_volume: return (355ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk 1073741824" returned: 0 in 0.050s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.213s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:40 user nova-compute[71317]: WARNING nova.compute.manager [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-changed-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Refreshing instance network info cache due to event network-changed-0c62956f-a2d8-4932-b48f-469eba99b8e6. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] Acquiring lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] Acquired lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.network.neutron [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Refreshing network info cache for port 0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.161s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Checking if we can resize image /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Cannot resize image /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.objects.instance [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lazy-loading 'migration_context' on Instance uuid 97392a47-53f0-4bde-977a-e2945332fe1f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Ensure instance console log exists: /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:40 user nova-compute[71317]: DEBUG nova.network.neutron [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Successfully created port: 32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG nova.network.neutron [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updated VIF entry in instance network info cache for port 0c62956f-a2d8-4932-b48f-469eba99b8e6. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG nova.network.neutron [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updating instance_info_cache with network_info: [{"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] Releasing lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-changed-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Refreshing instance network info cache due to event network-changed-0c62956f-a2d8-4932-b48f-469eba99b8e6. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] Acquiring lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] Acquired lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:22:41 user nova-compute[71317]: DEBUG nova.network.neutron [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Refreshing network info cache for port 0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.network.neutron [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updated VIF entry in instance network info cache for port 0c62956f-a2d8-4932-b48f-469eba99b8e6. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.network.neutron [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updating instance_info_cache with network_info: [{"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-53335f53-525c-4c31-8d64-a5fdf83c1e92 req-a71cb9c7-f629-487c-a460-16881658f6f5 service nova] Releasing lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.network.neutron [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Successfully updated port: 32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "refresh_cache-97392a47-53f0-4bde-977a-e2945332fe1f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquired lock "refresh_cache-97392a47-53f0-4bde-977a-e2945332fe1f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.network.neutron [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-eca3381b-b360-4cd2-8ccf-5b25ed8af697 req-eb108a2f-4620-4646-96e5-bcefcc37eeb5 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Received event network-changed-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-eca3381b-b360-4cd2-8ccf-5b25ed8af697 req-eb108a2f-4620-4646-96e5-bcefcc37eeb5 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Refreshing instance network info cache due to event network-changed-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-eca3381b-b360-4cd2-8ccf-5b25ed8af697 req-eb108a2f-4620-4646-96e5-bcefcc37eeb5 service nova] Acquiring lock "refresh_cache-97392a47-53f0-4bde-977a-e2945332fe1f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.network.neutron [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.network.neutron [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Updating instance_info_cache with network_info: [{"id": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "address": "fa:16:3e:6e:4a:74", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap32f6910c-73", "ovs_interfaceid": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Releasing lock "refresh_cache-97392a47-53f0-4bde-977a-e2945332fe1f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Instance network_info: |[{"id": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "address": "fa:16:3e:6e:4a:74", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap32f6910c-73", "ovs_interfaceid": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-eca3381b-b360-4cd2-8ccf-5b25ed8af697 req-eb108a2f-4620-4646-96e5-bcefcc37eeb5 service nova] Acquired lock "refresh_cache-97392a47-53f0-4bde-977a-e2945332fe1f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.network.neutron [req-eca3381b-b360-4cd2-8ccf-5b25ed8af697 req-eb108a2f-4620-4646-96e5-bcefcc37eeb5 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Refreshing network info cache for port 32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Start _get_guest_xml network_info=[{"id": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "address": "fa:16:3e:6e:4a:74", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap32f6910c-73", "ovs_interfaceid": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:22:42 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:22:42 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:22:38Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-929176901',display_name='tempest-VolumesSnapshotTestJSON-instance-929176901',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-929176901',id=19,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0848c9a67e2746e5b12b73920215b729',ramdisk_id='',reservation_id='r-p6ob5qid',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-1226436274',owner_user_name='tempest-VolumesSnapshotTestJSON-1226436274-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:22:40Z,user_data=None,user_id='e4a70281d6734c15b3c4b1b00e4b5d44',uuid=97392a47-53f0-4bde-977a-e2945332fe1f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "address": "fa:16:3e:6e:4a:74", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap32f6910c-73", "ovs_interfaceid": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Converting VIF {"id": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "address": "fa:16:3e:6e:4a:74", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap32f6910c-73", "ovs_interfaceid": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6e:4a:74,bridge_name='br-int',has_traffic_filtering=True,id=32f6910c-738f-4bbd-bd2a-a2d1b9bc261d,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap32f6910c-73') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.objects.instance [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lazy-loading 'pci_devices' on Instance uuid 97392a47-53f0-4bde-977a-e2945332fe1f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:22:42 user nova-compute[71317]:   <uuid>97392a47-53f0-4bde-977a-e2945332fe1f</uuid>
Apr 14 04:22:42 user nova-compute[71317]:   <name>instance-00000013</name>
Apr 14 04:22:42 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:22:42 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:22:42 user nova-compute[71317]:   <metadata>
Apr 14 04:22:42 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:22:42 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:22:42 user nova-compute[71317]:       <nova:name>tempest-VolumesSnapshotTestJSON-instance-929176901</nova:name>
Apr 14 04:22:42 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:22:42</nova:creationTime>
Apr 14 04:22:42 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:22:42 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:22:42 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:22:42 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:22:42 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:22:42 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:22:42 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:22:42 user nova-compute[71317]:       <nova:owner>
Apr 14 04:22:42 user nova-compute[71317]:         <nova:user uuid="e4a70281d6734c15b3c4b1b00e4b5d44">tempest-VolumesSnapshotTestJSON-1226436274-project-member</nova:user>
Apr 14 04:22:42 user nova-compute[71317]:         <nova:project uuid="0848c9a67e2746e5b12b73920215b729">tempest-VolumesSnapshotTestJSON-1226436274</nova:project>
Apr 14 04:22:42 user nova-compute[71317]:       </nova:owner>
Apr 14 04:22:42 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:22:42 user nova-compute[71317]:       <nova:ports>
Apr 14 04:22:42 user nova-compute[71317]:         <nova:port uuid="32f6910c-738f-4bbd-bd2a-a2d1b9bc261d">
Apr 14 04:22:42 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 04:22:42 user nova-compute[71317]:         </nova:port>
Apr 14 04:22:42 user nova-compute[71317]:       </nova:ports>
Apr 14 04:22:42 user nova-compute[71317]:     </nova:instance>
Apr 14 04:22:42 user nova-compute[71317]:   </metadata>
Apr 14 04:22:42 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:22:42 user nova-compute[71317]:     <system>
Apr 14 04:22:42 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:22:42 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:22:42 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:22:42 user nova-compute[71317]:       <entry name="serial">97392a47-53f0-4bde-977a-e2945332fe1f</entry>
Apr 14 04:22:42 user nova-compute[71317]:       <entry name="uuid">97392a47-53f0-4bde-977a-e2945332fe1f</entry>
Apr 14 04:22:42 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:22:42 user nova-compute[71317]:     </system>
Apr 14 04:22:42 user nova-compute[71317]:   </sysinfo>
Apr 14 04:22:42 user nova-compute[71317]:   <os>
Apr 14 04:22:42 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:22:42 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:22:42 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:22:42 user nova-compute[71317]:   </os>
Apr 14 04:22:42 user nova-compute[71317]:   <features>
Apr 14 04:22:42 user nova-compute[71317]:     <acpi/>
Apr 14 04:22:42 user nova-compute[71317]:     <apic/>
Apr 14 04:22:42 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:22:42 user nova-compute[71317]:   </features>
Apr 14 04:22:42 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:22:42 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:22:42 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:22:42 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:22:42 user nova-compute[71317]:   </clock>
Apr 14 04:22:42 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:22:42 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:22:42 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:22:42 user nova-compute[71317]:   </cpu>
Apr 14 04:22:42 user nova-compute[71317]:   <devices>
Apr 14 04:22:42 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:22:42 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:22:42 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk"/>
Apr 14 04:22:42 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:22:42 user nova-compute[71317]:     </disk>
Apr 14 04:22:42 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:22:42 user nova-compute[71317]:       <mac address="fa:16:3e:6e:4a:74"/>
Apr 14 04:22:42 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:22:42 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:22:42 user nova-compute[71317]:       <target dev="tap32f6910c-73"/>
Apr 14 04:22:42 user nova-compute[71317]:     </interface>
Apr 14 04:22:42 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:22:42 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/console.log" append="off"/>
Apr 14 04:22:42 user nova-compute[71317]:     </serial>
Apr 14 04:22:42 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:22:42 user nova-compute[71317]:     <video>
Apr 14 04:22:42 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:22:42 user nova-compute[71317]:     </video>
Apr 14 04:22:42 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:22:42 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:22:42 user nova-compute[71317]:     </rng>
Apr 14 04:22:42 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:22:42 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:22:42 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:22:42 user nova-compute[71317]:     </memballoon>
Apr 14 04:22:42 user nova-compute[71317]:   </devices>
Apr 14 04:22:42 user nova-compute[71317]: </domain>
Apr 14 04:22:42 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:22:38Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-929176901',display_name='tempest-VolumesSnapshotTestJSON-instance-929176901',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-929176901',id=19,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0848c9a67e2746e5b12b73920215b729',ramdisk_id='',reservation_id='r-p6ob5qid',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-1226436274',owner_user_name='tempest-VolumesSnapshotTestJSON-1226436274-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:22:40Z,user_data=None,user_id='e4a70281d6734c15b3c4b1b00e4b5d44',uuid=97392a47-53f0-4bde-977a-e2945332fe1f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "address": "fa:16:3e:6e:4a:74", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap32f6910c-73", "ovs_interfaceid": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Converting VIF {"id": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "address": "fa:16:3e:6e:4a:74", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap32f6910c-73", "ovs_interfaceid": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6e:4a:74,bridge_name='br-int',has_traffic_filtering=True,id=32f6910c-738f-4bbd-bd2a-a2d1b9bc261d,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap32f6910c-73') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG os_vif [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6e:4a:74,bridge_name='br-int',has_traffic_filtering=True,id=32f6910c-738f-4bbd-bd2a-a2d1b9bc261d,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap32f6910c-73') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap32f6910c-73, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap32f6910c-73, col_values=(('external_ids', {'iface-id': '32f6910c-738f-4bbd-bd2a-a2d1b9bc261d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6e:4a:74', 'vm-uuid': '97392a47-53f0-4bde-977a-e2945332fe1f'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-9673e2b5-69d3-4b19-ab42-c144e6d67ca1 req-a5e8504f-7d67-44a0-8383-814e20ff8fd4 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9673e2b5-69d3-4b19-ab42-c144e6d67ca1 req-a5e8504f-7d67-44a0-8383-814e20ff8fd4 service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9673e2b5-69d3-4b19-ab42-c144e6d67ca1 req-a5e8504f-7d67-44a0-8383-814e20ff8fd4 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9673e2b5-69d3-4b19-ab42-c144e6d67ca1 req-a5e8504f-7d67-44a0-8383-814e20ff8fd4 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-9673e2b5-69d3-4b19-ab42-c144e6d67ca1 req-a5e8504f-7d67-44a0-8383-814e20ff8fd4 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:42 user nova-compute[71317]: WARNING nova.compute.manager [req-9673e2b5-69d3-4b19-ab42-c144e6d67ca1 req-a5e8504f-7d67-44a0-8383-814e20ff8fd4 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-9673e2b5-69d3-4b19-ab42-c144e6d67ca1 req-a5e8504f-7d67-44a0-8383-814e20ff8fd4 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9673e2b5-69d3-4b19-ab42-c144e6d67ca1 req-a5e8504f-7d67-44a0-8383-814e20ff8fd4 service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9673e2b5-69d3-4b19-ab42-c144e6d67ca1 req-a5e8504f-7d67-44a0-8383-814e20ff8fd4 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9673e2b5-69d3-4b19-ab42-c144e6d67ca1 req-a5e8504f-7d67-44a0-8383-814e20ff8fd4 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-9673e2b5-69d3-4b19-ab42-c144e6d67ca1 req-a5e8504f-7d67-44a0-8383-814e20ff8fd4 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:42 user nova-compute[71317]: WARNING nova.compute.manager [req-9673e2b5-69d3-4b19-ab42-c144e6d67ca1 req-a5e8504f-7d67-44a0-8383-814e20ff8fd4 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:22:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:42 user nova-compute[71317]: INFO os_vif [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6e:4a:74,bridge_name='br-int',has_traffic_filtering=True,id=32f6910c-738f-4bbd-bd2a-a2d1b9bc261d,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap32f6910c-73')
Apr 14 04:22:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:22:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] No VIF found with MAC fa:16:3e:6e:4a:74, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:22:43 user nova-compute[71317]: DEBUG nova.network.neutron [req-eca3381b-b360-4cd2-8ccf-5b25ed8af697 req-eb108a2f-4620-4646-96e5-bcefcc37eeb5 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Updated VIF entry in instance network info cache for port 32f6910c-738f-4bbd-bd2a-a2d1b9bc261d. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:22:43 user nova-compute[71317]: DEBUG nova.network.neutron [req-eca3381b-b360-4cd2-8ccf-5b25ed8af697 req-eb108a2f-4620-4646-96e5-bcefcc37eeb5 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Updating instance_info_cache with network_info: [{"id": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "address": "fa:16:3e:6e:4a:74", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap32f6910c-73", "ovs_interfaceid": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-eca3381b-b360-4cd2-8ccf-5b25ed8af697 req-eb108a2f-4620-4646-96e5-bcefcc37eeb5 service nova] Releasing lock "refresh_cache-97392a47-53f0-4bde-977a-e2945332fe1f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:22:43 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:22:43 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446163.1887665, cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:22:43 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] VM Resumed (Lifecycle Event)
Apr 14 04:22:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-342ecc7a-d4c1-4ea1-bc76-9432521f482c tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:22:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:22:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:22:43 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 14 04:22:43 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446163.1950881, cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:22:43 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] VM Started (Lifecycle Event)
Apr 14 04:22:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:22:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:22:43 user nova-compute[71317]: DEBUG nova.objects.instance [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lazy-loading 'flavor' on Instance uuid d2e6cf32-080b-47b2-b4db-2b776ba85b38 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fe53da21-462c-4f86-9fee-2bb335f82587 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 4.594s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:44 user nova-compute[71317]: DEBUG nova.compute.manager [req-74fc59ad-ec76-4abe-b962-29da1ea900d9 req-c609f371-cd63-4790-8ddd-85bdc6a2e125 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Received event network-vif-plugged-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-74fc59ad-ec76-4abe-b962-29da1ea900d9 req-c609f371-cd63-4790-8ddd-85bdc6a2e125 service nova] Acquiring lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-74fc59ad-ec76-4abe-b962-29da1ea900d9 req-c609f371-cd63-4790-8ddd-85bdc6a2e125 service nova] Lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-74fc59ad-ec76-4abe-b962-29da1ea900d9 req-c609f371-cd63-4790-8ddd-85bdc6a2e125 service nova] Lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:44 user nova-compute[71317]: DEBUG nova.compute.manager [req-74fc59ad-ec76-4abe-b962-29da1ea900d9 req-c609f371-cd63-4790-8ddd-85bdc6a2e125 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] No waiting events found dispatching network-vif-plugged-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:44 user nova-compute[71317]: WARNING nova.compute.manager [req-74fc59ad-ec76-4abe-b962-29da1ea900d9 req-c609f371-cd63-4790-8ddd-85bdc6a2e125 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Received unexpected event network-vif-plugged-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d for instance with vm_state building and task_state spawning.
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446166.5956156, 97392a47-53f0-4bde-977a-e2945332fe1f => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:22:46 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] VM Resumed (Lifecycle Event)
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:22:46 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Instance spawned successfully.
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:22:46 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446166.5992324, 97392a47-53f0-4bde-977a-e2945332fe1f => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:22:46 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] VM Started (Lifecycle Event)
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:22:46 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:22:46 user nova-compute[71317]: INFO nova.compute.manager [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Took 6.53 seconds to spawn the instance on the hypervisor.
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-f324fef4-cd81-45bb-bed5-1c96a4c75c63 req-b5e799c8-7311-4aae-a5ca-d559f05a87d3 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Received event network-vif-plugged-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f324fef4-cd81-45bb-bed5-1c96a4c75c63 req-b5e799c8-7311-4aae-a5ca-d559f05a87d3 service nova] Acquiring lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f324fef4-cd81-45bb-bed5-1c96a4c75c63 req-b5e799c8-7311-4aae-a5ca-d559f05a87d3 service nova] Lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f324fef4-cd81-45bb-bed5-1c96a4c75c63 req-b5e799c8-7311-4aae-a5ca-d559f05a87d3 service nova] Lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-f324fef4-cd81-45bb-bed5-1c96a4c75c63 req-b5e799c8-7311-4aae-a5ca-d559f05a87d3 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] No waiting events found dispatching network-vif-plugged-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:46 user nova-compute[71317]: WARNING nova.compute.manager [req-f324fef4-cd81-45bb-bed5-1c96a4c75c63 req-b5e799c8-7311-4aae-a5ca-d559f05a87d3 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Received unexpected event network-vif-plugged-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d for instance with vm_state active and task_state None.
Apr 14 04:22:46 user nova-compute[71317]: INFO nova.compute.manager [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Took 7.41 seconds to build instance.
Apr 14 04:22:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7e085d7f-3da9-4a0d-b9c8-8d2b384fa2fd tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "97392a47-53f0-4bde-977a-e2945332fe1f" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.530s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Acquiring lock "46a5fd77-abec-4ab3-b519-7c649e63459c" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:22:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:55 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:22:55 user nova-compute[71317]: INFO nova.compute.claims [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Claim successful on node user
Apr 14 04:22:55 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.713s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG nova.network.neutron [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:22:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:22:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:22:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Creating image(s)
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Acquiring lock "/opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "/opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "/opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG nova.policy [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '275086d3eec449678b6b5ba6f3f887b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f45f01678f747268221c00f75924c69', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.181s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.175s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk 1073741824" returned: 0 in 0.078s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.263s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.153s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Checking if we can resize image /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:56 user nova-compute[71317]: INFO nova.compute.manager [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Terminating instance
Apr 14 04:22:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Cannot resize image /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.objects.instance [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lazy-loading 'migration_context' on Instance uuid 46a5fd77-abec-4ab3-b519-7c649e63459c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Ensure instance console log exists: /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-84cb6972-b9d7-4264-9d14-db7795e754cb req-4749b97c-1236-452a-952d-f0749f5be220 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-vif-unplugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-84cb6972-b9d7-4264-9d14-db7795e754cb req-4749b97c-1236-452a-952d-f0749f5be220 service nova] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-84cb6972-b9d7-4264-9d14-db7795e754cb req-4749b97c-1236-452a-952d-f0749f5be220 service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-84cb6972-b9d7-4264-9d14-db7795e754cb req-4749b97c-1236-452a-952d-f0749f5be220 service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-84cb6972-b9d7-4264-9d14-db7795e754cb req-4749b97c-1236-452a-952d-f0749f5be220 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] No waiting events found dispatching network-vif-unplugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-84cb6972-b9d7-4264-9d14-db7795e754cb req-4749b97c-1236-452a-952d-f0749f5be220 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-vif-unplugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.network.neutron [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Successfully created port: 534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:57 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Instance destroyed successfully.
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'resources' on Instance uuid 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:17:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-99049874',display_name='tempest-TestMinimumBasicScenario-server-99049874',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-99049874',id=13,image_ref='2921b7b4-17d1-4c56-9fbc-a30733db1472',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA++2BXIcEQ2v7bj136wrEst3TLhMf3Pwm5yFI2y5OTUcyHd3G9mX9nGbcm6RmV/qF9Miy1upxS4ewe90Jmk1PeD7f+WKuozsw6VgX059vi7WiBA0frzDuQmMNJN3b4XIg==',key_name='tempest-TestMinimumBasicScenario-1599913910',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:00Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0a60f60717b44f4ba904a949eabfe608',ramdisk_id='',reservation_id='r-n2gdtvof',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='2921b7b4-17d1-4c56-9fbc-a30733db1472',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1872250892',owner_user_name='tempest-TestMinimumBasicScenario-1872250892-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:20:16Z,user_data=None,user_id='982f167333954612a51911fc8d6c1ce9',uuid=4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converting VIF {"id": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "address": "fa:16:3e:0f:a8:af", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef7670a9-34", "ovs_interfaceid": "ef7670a9-3421-45de-a1cd-976ddb46f2a1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG os_vif [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapef7670a9-34, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:57 user nova-compute[71317]: INFO os_vif [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:0f:a8:af,bridge_name='br-int',has_traffic_filtering=True,id=ef7670a9-3421-45de-a1cd-976ddb46f2a1,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef7670a9-34')
Apr 14 04:22:57 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Deleting instance files /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e_del
Apr 14 04:22:57 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Deletion of /opt/stack/data/nova/instances/4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e_del complete
Apr 14 04:22:57 user nova-compute[71317]: INFO nova.compute.manager [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Took 1.03 seconds to destroy the instance on the hypervisor.
Apr 14 04:22:57 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:22:57 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Successfully updated port: 534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Acquiring lock "refresh_cache-46a5fd77-abec-4ab3-b519-7c649e63459c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Acquired lock "refresh_cache-46a5fd77-abec-4ab3-b519-7c649e63459c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:22:58 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Took 0.57 seconds to deallocate network for instance.
Apr 14 04:22:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-73555978-b842-4ecd-89d0-428d03be3c99 req-889cbcdf-4727-4afa-a945-992202d98df3 service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Received event network-changed-534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-73555978-b842-4ecd-89d0-428d03be3c99 req-889cbcdf-4727-4afa-a945-992202d98df3 service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Refreshing instance network info cache due to event network-changed-534d2295-4787-4bd1-9e53-4cf6acbf4a2b. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-73555978-b842-4ecd-89d0-428d03be3c99 req-889cbcdf-4727-4afa-a945-992202d98df3 service nova] Acquiring lock "refresh_cache-46a5fd77-abec-4ab3-b519-7c649e63459c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Updating instance_info_cache with network_info: [{"id": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "address": "fa:16:3e:9d:19:0d", "network": {"id": "b96b0659-2bac-40e7-b707-62f8a9fae7a7", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.87", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "d8b3c3fd68cb4219826c2a6fd8b21939", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap534d2295-47", "ovs_interfaceid": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Releasing lock "refresh_cache-46a5fd77-abec-4ab3-b519-7c649e63459c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Instance network_info: |[{"id": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "address": "fa:16:3e:9d:19:0d", "network": {"id": "b96b0659-2bac-40e7-b707-62f8a9fae7a7", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.87", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "d8b3c3fd68cb4219826c2a6fd8b21939", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap534d2295-47", "ovs_interfaceid": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-73555978-b842-4ecd-89d0-428d03be3c99 req-889cbcdf-4727-4afa-a945-992202d98df3 service nova] Acquired lock "refresh_cache-46a5fd77-abec-4ab3-b519-7c649e63459c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:22:58 user nova-compute[71317]: DEBUG nova.network.neutron [req-73555978-b842-4ecd-89d0-428d03be3c99 req-889cbcdf-4727-4afa-a945-992202d98df3 service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Refreshing network info cache for port 534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Start _get_guest_xml network_info=[{"id": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "address": "fa:16:3e:9d:19:0d", "network": {"id": "b96b0659-2bac-40e7-b707-62f8a9fae7a7", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.87", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "d8b3c3fd68cb4219826c2a6fd8b21939", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap534d2295-47", "ovs_interfaceid": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:22:59 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:22:59 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:22:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-1070759898',display_name='tempest-VolumesNegativeTest-instance-1070759898',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-1070759898',id=20,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3f45f01678f747268221c00f75924c69',ramdisk_id='',reservation_id='r-wncqohyy',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesNegativeTest-1984420239',owner_user_name='tempest-VolumesNegativeTest-1984420239-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:22:56Z,user_data=None,user_id='275086d3eec449678b6b5ba6f3f887b2',uuid=46a5fd77-abec-4ab3-b519-7c649e63459c,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "address": "fa:16:3e:9d:19:0d", "network": {"id": "b96b0659-2bac-40e7-b707-62f8a9fae7a7", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.87", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "d8b3c3fd68cb4219826c2a6fd8b21939", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap534d2295-47", "ovs_interfaceid": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Converting VIF {"id": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "address": "fa:16:3e:9d:19:0d", "network": {"id": "b96b0659-2bac-40e7-b707-62f8a9fae7a7", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.87", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "d8b3c3fd68cb4219826c2a6fd8b21939", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap534d2295-47", "ovs_interfaceid": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9d:19:0d,bridge_name='br-int',has_traffic_filtering=True,id=534d2295-4787-4bd1-9e53-4cf6acbf4a2b,network=Network(b96b0659-2bac-40e7-b707-62f8a9fae7a7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap534d2295-47') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.objects.instance [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lazy-loading 'pci_devices' on Instance uuid 46a5fd77-abec-4ab3-b519-7c649e63459c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.426s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:22:59 user nova-compute[71317]:   <uuid>46a5fd77-abec-4ab3-b519-7c649e63459c</uuid>
Apr 14 04:22:59 user nova-compute[71317]:   <name>instance-00000014</name>
Apr 14 04:22:59 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:22:59 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:22:59 user nova-compute[71317]:   <metadata>
Apr 14 04:22:59 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:22:59 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:22:59 user nova-compute[71317]:       <nova:name>tempest-VolumesNegativeTest-instance-1070759898</nova:name>
Apr 14 04:22:59 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:22:59</nova:creationTime>
Apr 14 04:22:59 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:22:59 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:22:59 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:22:59 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:22:59 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:22:59 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:22:59 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:22:59 user nova-compute[71317]:       <nova:owner>
Apr 14 04:22:59 user nova-compute[71317]:         <nova:user uuid="275086d3eec449678b6b5ba6f3f887b2">tempest-VolumesNegativeTest-1984420239-project-member</nova:user>
Apr 14 04:22:59 user nova-compute[71317]:         <nova:project uuid="3f45f01678f747268221c00f75924c69">tempest-VolumesNegativeTest-1984420239</nova:project>
Apr 14 04:22:59 user nova-compute[71317]:       </nova:owner>
Apr 14 04:22:59 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:22:59 user nova-compute[71317]:       <nova:ports>
Apr 14 04:22:59 user nova-compute[71317]:         <nova:port uuid="534d2295-4787-4bd1-9e53-4cf6acbf4a2b">
Apr 14 04:22:59 user nova-compute[71317]:           <nova:ip type="fixed" address="192.168.233.87" ipVersion="4"/>
Apr 14 04:22:59 user nova-compute[71317]:         </nova:port>
Apr 14 04:22:59 user nova-compute[71317]:       </nova:ports>
Apr 14 04:22:59 user nova-compute[71317]:     </nova:instance>
Apr 14 04:22:59 user nova-compute[71317]:   </metadata>
Apr 14 04:22:59 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:22:59 user nova-compute[71317]:     <system>
Apr 14 04:22:59 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:22:59 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:22:59 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:22:59 user nova-compute[71317]:       <entry name="serial">46a5fd77-abec-4ab3-b519-7c649e63459c</entry>
Apr 14 04:22:59 user nova-compute[71317]:       <entry name="uuid">46a5fd77-abec-4ab3-b519-7c649e63459c</entry>
Apr 14 04:22:59 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:22:59 user nova-compute[71317]:     </system>
Apr 14 04:22:59 user nova-compute[71317]:   </sysinfo>
Apr 14 04:22:59 user nova-compute[71317]:   <os>
Apr 14 04:22:59 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:22:59 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:22:59 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:22:59 user nova-compute[71317]:   </os>
Apr 14 04:22:59 user nova-compute[71317]:   <features>
Apr 14 04:22:59 user nova-compute[71317]:     <acpi/>
Apr 14 04:22:59 user nova-compute[71317]:     <apic/>
Apr 14 04:22:59 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:22:59 user nova-compute[71317]:   </features>
Apr 14 04:22:59 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:22:59 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:22:59 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:22:59 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:22:59 user nova-compute[71317]:   </clock>
Apr 14 04:22:59 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:22:59 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:22:59 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:22:59 user nova-compute[71317]:   </cpu>
Apr 14 04:22:59 user nova-compute[71317]:   <devices>
Apr 14 04:22:59 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:22:59 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:22:59 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk"/>
Apr 14 04:22:59 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:22:59 user nova-compute[71317]:     </disk>
Apr 14 04:22:59 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:22:59 user nova-compute[71317]:       <mac address="fa:16:3e:9d:19:0d"/>
Apr 14 04:22:59 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:22:59 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:22:59 user nova-compute[71317]:       <target dev="tap534d2295-47"/>
Apr 14 04:22:59 user nova-compute[71317]:     </interface>
Apr 14 04:22:59 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:22:59 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/console.log" append="off"/>
Apr 14 04:22:59 user nova-compute[71317]:     </serial>
Apr 14 04:22:59 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:22:59 user nova-compute[71317]:     <video>
Apr 14 04:22:59 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:22:59 user nova-compute[71317]:     </video>
Apr 14 04:22:59 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:22:59 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:22:59 user nova-compute[71317]:     </rng>
Apr 14 04:22:59 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:22:59 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:22:59 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:22:59 user nova-compute[71317]:     </memballoon>
Apr 14 04:22:59 user nova-compute[71317]:   </devices>
Apr 14 04:22:59 user nova-compute[71317]: </domain>
Apr 14 04:22:59 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:22:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-1070759898',display_name='tempest-VolumesNegativeTest-instance-1070759898',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-1070759898',id=20,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3f45f01678f747268221c00f75924c69',ramdisk_id='',reservation_id='r-wncqohyy',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesNegativeTest-1984420239',owner_user_name='tempest-VolumesNegativeTest-1984420239-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:22:56Z,user_data=None,user_id='275086d3eec449678b6b5ba6f3f887b2',uuid=46a5fd77-abec-4ab3-b519-7c649e63459c,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "address": "fa:16:3e:9d:19:0d", "network": {"id": "b96b0659-2bac-40e7-b707-62f8a9fae7a7", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.87", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "d8b3c3fd68cb4219826c2a6fd8b21939", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap534d2295-47", "ovs_interfaceid": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Converting VIF {"id": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "address": "fa:16:3e:9d:19:0d", "network": {"id": "b96b0659-2bac-40e7-b707-62f8a9fae7a7", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.87", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "d8b3c3fd68cb4219826c2a6fd8b21939", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap534d2295-47", "ovs_interfaceid": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9d:19:0d,bridge_name='br-int',has_traffic_filtering=True,id=534d2295-4787-4bd1-9e53-4cf6acbf4a2b,network=Network(b96b0659-2bac-40e7-b707-62f8a9fae7a7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap534d2295-47') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG os_vif [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9d:19:0d,bridge_name='br-int',has_traffic_filtering=True,id=534d2295-4787-4bd1-9e53-4cf6acbf4a2b,network=Network(b96b0659-2bac-40e7-b707-62f8a9fae7a7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap534d2295-47') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap534d2295-47, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap534d2295-47, col_values=(('external_ids', {'iface-id': '534d2295-4787-4bd1-9e53-4cf6acbf4a2b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:9d:19:0d', 'vm-uuid': '46a5fd77-abec-4ab3-b519-7c649e63459c'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:22:59 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Deleted allocations for instance 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e
Apr 14 04:22:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:22:59 user nova-compute[71317]: INFO os_vif [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9d:19:0d,bridge_name='br-int',has_traffic_filtering=True,id=534d2295-4787-4bd1-9e53-4cf6acbf4a2b,network=Network(b96b0659-2bac-40e7-b707-62f8a9fae7a7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap534d2295-47')
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] No VIF found with MAC fa:16:3e:9d:19:0d, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0eef3cf5-0432-497f-9b70-274ee8688e6e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.321s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-6f417b1e-e458-4397-bc27-87a4d378ea53 req-4f57f0fa-d6c9-4f2f-9a0d-ce0291031717 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6f417b1e-e458-4397-bc27-87a4d378ea53 req-4f57f0fa-d6c9-4f2f-9a0d-ce0291031717 service nova] Acquiring lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6f417b1e-e458-4397-bc27-87a4d378ea53 req-4f57f0fa-d6c9-4f2f-9a0d-ce0291031717 service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6f417b1e-e458-4397-bc27-87a4d378ea53 req-4f57f0fa-d6c9-4f2f-9a0d-ce0291031717 service nova] Lock "4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-6f417b1e-e458-4397-bc27-87a4d378ea53 req-4f57f0fa-d6c9-4f2f-9a0d-ce0291031717 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] No waiting events found dispatching network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:22:59 user nova-compute[71317]: WARNING nova.compute.manager [req-6f417b1e-e458-4397-bc27-87a4d378ea53 req-4f57f0fa-d6c9-4f2f-9a0d-ce0291031717 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received unexpected event network-vif-plugged-ef7670a9-3421-45de-a1cd-976ddb46f2a1 for instance with vm_state deleted and task_state None.
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-6f417b1e-e458-4397-bc27-87a4d378ea53 req-4f57f0fa-d6c9-4f2f-9a0d-ce0291031717 service nova] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Received event network-vif-deleted-ef7670a9-3421-45de-a1cd-976ddb46f2a1 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-73555978-b842-4ecd-89d0-428d03be3c99 req-889cbcdf-4727-4afa-a945-992202d98df3 service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Updated VIF entry in instance network info cache for port 534d2295-4787-4bd1-9e53-4cf6acbf4a2b. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-73555978-b842-4ecd-89d0-428d03be3c99 req-889cbcdf-4727-4afa-a945-992202d98df3 service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Updating instance_info_cache with network_info: [{"id": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "address": "fa:16:3e:9d:19:0d", "network": {"id": "b96b0659-2bac-40e7-b707-62f8a9fae7a7", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.87", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "d8b3c3fd68cb4219826c2a6fd8b21939", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap534d2295-47", "ovs_interfaceid": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:22:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-73555978-b842-4ecd-89d0-428d03be3c99 req-889cbcdf-4727-4afa-a945-992202d98df3 service nova] Releasing lock "refresh_cache-46a5fd77-abec-4ab3-b519-7c649e63459c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:23:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e3e81ad-17d2-41e8-8081-7c1753ec074c req-400314f4-d6d1-47a1-bd13-59669743c37b service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Received event network-vif-plugged-534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e3e81ad-17d2-41e8-8081-7c1753ec074c req-400314f4-d6d1-47a1-bd13-59669743c37b service nova] Acquiring lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e3e81ad-17d2-41e8-8081-7c1753ec074c req-400314f4-d6d1-47a1-bd13-59669743c37b service nova] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2e3e81ad-17d2-41e8-8081-7c1753ec074c req-400314f4-d6d1-47a1-bd13-59669743c37b service nova] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-2e3e81ad-17d2-41e8-8081-7c1753ec074c req-400314f4-d6d1-47a1-bd13-59669743c37b service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] No waiting events found dispatching network-vif-plugged-534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:23:00 user nova-compute[71317]: WARNING nova.compute.manager [req-2e3e81ad-17d2-41e8-8081-7c1753ec074c req-400314f4-d6d1-47a1-bd13-59669743c37b service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Received unexpected event network-vif-plugged-534d2295-4787-4bd1-9e53-4cf6acbf4a2b for instance with vm_state building and task_state spawning.
Apr 14 04:23:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446182.829927, 46a5fd77-abec-4ab3-b519-7c649e63459c => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:23:02 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] VM Resumed (Lifecycle Event)
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:23:02 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Instance spawned successfully.
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:02 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446182.8309927, 46a5fd77-abec-4ab3-b519-7c649e63459c => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:23:02 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] VM Started (Lifecycle Event)
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-1a431e6c-538f-44c2-bb52-2e5d07cd6203 req-54691ba7-e93c-4fe5-b471-e84e8deebe78 service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Received event network-vif-plugged-534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1a431e6c-538f-44c2-bb52-2e5d07cd6203 req-54691ba7-e93c-4fe5-b471-e84e8deebe78 service nova] Acquiring lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1a431e6c-538f-44c2-bb52-2e5d07cd6203 req-54691ba7-e93c-4fe5-b471-e84e8deebe78 service nova] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1a431e6c-538f-44c2-bb52-2e5d07cd6203 req-54691ba7-e93c-4fe5-b471-e84e8deebe78 service nova] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-1a431e6c-538f-44c2-bb52-2e5d07cd6203 req-54691ba7-e93c-4fe5-b471-e84e8deebe78 service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] No waiting events found dispatching network-vif-plugged-534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:23:02 user nova-compute[71317]: WARNING nova.compute.manager [req-1a431e6c-538f-44c2-bb52-2e5d07cd6203 req-54691ba7-e93c-4fe5-b471-e84e8deebe78 service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Received unexpected event network-vif-plugged-534d2295-4787-4bd1-9e53-4cf6acbf4a2b for instance with vm_state building and task_state spawning.
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:23:02 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:23:02 user nova-compute[71317]: INFO nova.compute.manager [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Took 6.70 seconds to spawn the instance on the hypervisor.
Apr 14 04:23:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:03 user nova-compute[71317]: INFO nova.compute.manager [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Took 7.75 seconds to build instance.
Apr 14 04:23:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-379c21e7-6515-4227-a781-e31f0bf4c638 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.858s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:07 user nova-compute[71317]: INFO nova.compute.manager [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] instance snapshotting
Apr 14 04:23:07 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Beginning live snapshot process
Apr 14 04:23:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json -f qcow2 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json -f qcow2" returned: 0 in 0.151s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json -f qcow2 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json -f qcow2" returned: 0 in 0.130s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.128s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpinp0hgqh/a092418db6ac407ba0f0d6125b45d9e2.delta 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpinp0hgqh/a092418db6ac407ba0f0d6125b45d9e2.delta 1073741824" returned: 0 in 0.049s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:08 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Quiescing instance not available: QEMU guest agent is not enabled.
Apr 14 04:23:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] COPY block job progress, current cursor: 0 final cursor: 43778048 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:23:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] COPY block job progress, current cursor: 43778048 final cursor: 43778048 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:23:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Skipping quiescing instance: QEMU guest agent is not enabled.
Apr 14 04:23:09 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:23:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpinp0hgqh/a092418db6ac407ba0f0d6125b45d9e2.delta /opt/stack/data/nova/instances/snapshots/tmpinp0hgqh/a092418db6ac407ba0f0d6125b45d9e2 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpinp0hgqh/a092418db6ac407ba0f0d6125b45d9e2.delta /opt/stack/data/nova/instances/snapshots/tmpinp0hgqh/a092418db6ac407ba0f0d6125b45d9e2" returned: 0 in 0.659s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:10 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Snapshot extracted, beginning image upload
Apr 14 04:23:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:11 user nova-compute[71317]: INFO nova.compute.manager [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Detaching volume 3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5
Apr 14 04:23:11 user nova-compute[71317]: INFO nova.virt.block_device [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Attempting to driver detach volume 3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 from mountpoint /dev/vdb
Apr 14 04:23:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Attempting to detach device vdb from instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:23:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:23:11 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:23:11 user nova-compute[71317]:   <source dev="/dev/sdh"/>
Apr 14 04:23:11 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:23:11 user nova-compute[71317]:   <serial>3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5</serial>
Apr 14 04:23:11 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:23:11 user nova-compute[71317]: </disk>
Apr 14 04:23:11 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:23:11 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Successfully detached device vdb from instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 from the persistent domain config.
Apr 14 04:23:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:23:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:23:11 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:23:11 user nova-compute[71317]:   <source dev="/dev/sdh"/>
Apr 14 04:23:11 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:23:11 user nova-compute[71317]:   <serial>3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5</serial>
Apr 14 04:23:11 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:23:11 user nova-compute[71317]: </disk>
Apr 14 04:23:11 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446192.2858477, cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:23:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Successfully detached device vdb from instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 from the live domain config.
Apr 14 04:23:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SvtMEfzm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:23:12 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[adec58cc-1dba-4551-8d64-a9bd71b6c5db]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:23:12 user nova-compute[71317]: No targets found.
Apr 14 04:23:12 user nova-compute[71317]: iSNS:
Apr 14 04:23:12 user nova-compute[71317]: No targets found.
Apr 14 04:23:12 user nova-compute[71317]: STATIC:
Apr 14 04:23:12 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5
Apr 14 04:23:12 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:23:12 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:23:12 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e
Apr 14 04:23:12 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:23:12 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:23:12 user nova-compute[71317]: FIRMWARE:
Apr 14 04:23:12 user nova-compute[71317]: No targets found.
Apr 14 04:23:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:23:12 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:23:12 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[52027903-85b2-4e2e-b09a-535a416d284d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6ad3d22-d385-4f41-bafe-7e13f10b463a]: (4, ('tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:23:12 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:23:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsi session list stdout=tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 (non-flash)
Apr 14 04:23:12 user nova-compute[71317]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:23:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c41240d0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5'): ({'sdh'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Removing single pathed devices sdh {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa8361b8-c75c-46ff-8cd4-21d7ed370069]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0900ec52-0873-4b8d-a530-1ff3eadcb647]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Flushing IO for device /dev/sdh {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3311a48d-8554-44b4-b96f-4ee8e6f9b876]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.039s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7228c42c-10f5-4585-976f-902228c02dc8]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] SCSI volumes sdh have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[882a9845-87d2-4984-9801-36a864c70719]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --logout" returned: 0 in 0.061s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba928814-0371-41ec-8eb1-6e1bc88bd020]: (4, ('Logging out of session [sid: 19, target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5, portal: 172.16.0.220,3260]\nLogout of [sid: 19, target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 19, target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5, portal: 172.16.0.220,3260]
Apr 14 04:23:12 user nova-compute[71317]: Logout of [sid: 19, target: iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5, portal: 172.16.0.220,3260] successful.
Apr 14 04:23:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3c65cdc6-8973-4e7f-a2e0-6d8ff88aa7f5 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[590bd39d-4cac-4ce8-9e6d-cd88db0770bf]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.339s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] <== disconnect_volume: return (341ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:12 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446177.8217592, 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:23:12 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] VM Stopped (Lifecycle Event)
Apr 14 04:23:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-595e59fc-816d-4435-b479-36927f9d7ab1 None None] [instance: 4d5e8a8f-7f0d-453e-90a8-ee18ed034f1e] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Snapshot image upload complete
Apr 14 04:23:13 user nova-compute[71317]: INFO nova.compute.manager [None req-6c6ab995-9459-491b-8dbb-e3fb0e73d68a tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Took 5.45 seconds to snapshot the instance on the hypervisor.
Apr 14 04:23:13 user nova-compute[71317]: DEBUG nova.objects.instance [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'flavor' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:23:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fce9499b-e331-4ce2-a88d-832c24a656d5 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.186s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:19 user nova-compute[71317]: INFO nova.compute.manager [None req-53c3e6fb-5c8b-49b6-8dfc-88ac03e519e9 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Unrescuing
Apr 14 04:23:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-53c3e6fb-5c8b-49b6-8dfc-88ac03e519e9 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:23:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-53c3e6fb-5c8b-49b6-8dfc-88ac03e519e9 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquired lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:23:19 user nova-compute[71317]: DEBUG nova.network.neutron [None req-53c3e6fb-5c8b-49b6-8dfc-88ac03e519e9 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:23:20 user nova-compute[71317]: DEBUG nova.network.neutron [None req-53c3e6fb-5c8b-49b6-8dfc-88ac03e519e9 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Updating instance_info_cache with network_info: [{"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:23:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-53c3e6fb-5c8b-49b6-8dfc-88ac03e519e9 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Releasing lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:23:20 user nova-compute[71317]: DEBUG nova.objects.instance [None req-53c3e6fb-5c8b-49b6-8dfc-88ac03e519e9 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'flavor' on Instance uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:23:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-67d22136-30ea-4b70-91ca-460be89da18b req-f6389650-fe58-417f-94a6-f612d54b2a9d service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-unplugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-67d22136-30ea-4b70-91ca-460be89da18b req-f6389650-fe58-417f-94a6-f612d54b2a9d service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-67d22136-30ea-4b70-91ca-460be89da18b req-f6389650-fe58-417f-94a6-f612d54b2a9d service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-67d22136-30ea-4b70-91ca-460be89da18b req-f6389650-fe58-417f-94a6-f612d54b2a9d service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-67d22136-30ea-4b70-91ca-460be89da18b req-f6389650-fe58-417f-94a6-f612d54b2a9d service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-unplugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:23:20 user nova-compute[71317]: WARNING nova.compute.manager [req-67d22136-30ea-4b70-91ca-460be89da18b req-f6389650-fe58-417f-94a6-f612d54b2a9d service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-unplugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:23:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:21 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Instance destroyed successfully.
Apr 14 04:23:22 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_power_states {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:23:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-c2c56ebb-58f4-48c0-8b8d-23dd813de4fc req-1f10568d-b494-4fe3-a772-651b29b7f460 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c2c56ebb-58f4-48c0-8b8d-23dd813de4fc req-1f10568d-b494-4fe3-a772-651b29b7f460 service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c2c56ebb-58f4-48c0-8b8d-23dd813de4fc req-1f10568d-b494-4fe3-a772-651b29b7f460 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c2c56ebb-58f4-48c0-8b8d-23dd813de4fc req-1f10568d-b494-4fe3-a772-651b29b7f460 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-c2c56ebb-58f4-48c0-8b8d-23dd813de4fc req-1f10568d-b494-4fe3-a772-651b29b7f460 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:23:22 user nova-compute[71317]: WARNING nova.compute.manager [req-c2c56ebb-58f4-48c0-8b8d-23dd813de4fc req-1f10568d-b494-4fe3-a772-651b29b7f460 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:23:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 9b532a44-20b7-4590-babe-684c49598a62 {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid d2e6cf32-080b-47b2-b4db-2b776ba85b38 {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 97392a47-53f0-4bde-977a-e2945332fe1f {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 46a5fd77-abec-4ab3-b519-7c649e63459c {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "9b532a44-20b7-4590-babe-684c49598a62" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "9b532a44-20b7-4590-babe-684c49598a62" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:23 user nova-compute[71317]: INFO nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "97392a47-53f0-4bde-977a-e2945332fe1f" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "97392a47-53f0-4bde-977a-e2945332fe1f" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "46a5fd77-abec-4ab3-b519-7c649e63459c" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.079s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.079s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.085s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "9b532a44-20b7-4590-babe-684c49598a62" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.090s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.085s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.087s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "97392a47-53f0-4bde-977a-e2945332fe1f" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.091s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.092s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.099s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:24 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:23:24 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:23:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:23:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:23:24 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:23:24 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updating instance_info_cache with network_info: [{"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:23:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:23:24 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:23:24 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:23:24 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-47b7391b-d171-407d-a011-60b46481a61d req-f7da9e5f-7d0a-4af4-aee4-419a96de3649 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-47b7391b-d171-407d-a011-60b46481a61d req-f7da9e5f-7d0a-4af4-aee4-419a96de3649 service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-47b7391b-d171-407d-a011-60b46481a61d req-f7da9e5f-7d0a-4af4-aee4-419a96de3649 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-47b7391b-d171-407d-a011-60b46481a61d req-f7da9e5f-7d0a-4af4-aee4-419a96de3649 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-47b7391b-d171-407d-a011-60b46481a61d req-f7da9e5f-7d0a-4af4-aee4-419a96de3649 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:23:25 user nova-compute[71317]: WARNING nova.compute.manager [req-47b7391b-d171-407d-a011-60b46481a61d req-f7da9e5f-7d0a-4af4-aee4-419a96de3649 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:23:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-47b7391b-d171-407d-a011-60b46481a61d req-f7da9e5f-7d0a-4af4-aee4-419a96de3649 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-47b7391b-d171-407d-a011-60b46481a61d req-f7da9e5f-7d0a-4af4-aee4-419a96de3649 service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-47b7391b-d171-407d-a011-60b46481a61d req-f7da9e5f-7d0a-4af4-aee4-419a96de3649 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-47b7391b-d171-407d-a011-60b46481a61d req-f7da9e5f-7d0a-4af4-aee4-419a96de3649 service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-47b7391b-d171-407d-a011-60b46481a61d req-f7da9e5f-7d0a-4af4-aee4-419a96de3649 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:23:25 user nova-compute[71317]: WARNING nova.compute.manager [req-47b7391b-d171-407d-a011-60b46481a61d req-f7da9e5f-7d0a-4af4-aee4-419a96de3649 service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:23:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-53c3e6fb-5c8b-49b6-8dfc-88ac03e519e9 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446205.0346174, 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:23:25 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] VM Resumed (Lifecycle Event)
Apr 14 04:23:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:23:25 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 14 04:23:25 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446205.0355833, 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:23:25 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] VM Started (Lifecycle Event)
Apr 14 04:23:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:23:25 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:23:26 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:23:26 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:23:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:28 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:23:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:29 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:23:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:23:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:30 user nova-compute[71317]: DEBUG nova.compute.manager [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:23:31 user nova-compute[71317]: INFO nova.compute.claims [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Claim successful on node user
Apr 14 04:23:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.454s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG nova.network.neutron [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:23:31 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:23:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG nova.policy [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd2561bbc0c6d447da333ffdbb9fdc5f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d389bb78d2e40b9b07d2d3ecdb7566b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:23:31 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Creating image(s)
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "/opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "/opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "/opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk 1073741824" returned: 0 in 0.068s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.215s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.162s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Checking if we can resize image /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Cannot resize image /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG nova.objects.instance [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lazy-loading 'migration_context' on Instance uuid 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Ensure instance console log exists: /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:32 user nova-compute[71317]: INFO nova.compute.manager [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Detaching volume 3e0082ad-1ad0-4980-a066-6017f65fce9e
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.172s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:32 user nova-compute[71317]: INFO nova.virt.block_device [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Attempting to driver detach volume 3e0082ad-1ad0-4980-a066-6017f65fce9e from mountpoint /dev/vdb
Apr 14 04:23:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Attempting to detach device vdb from instance a0a20aeb-ed06-4c65-b87d-74f8f825457d from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:23:32 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:23:32 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:23:32 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:23:32 user nova-compute[71317]:   <serial>3e0082ad-1ad0-4980-a066-6017f65fce9e</serial>
Apr 14 04:23:32 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 04:23:32 user nova-compute[71317]: </disk>
Apr 14 04:23:32 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:23:32 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully detached device vdb from instance a0a20aeb-ed06-4c65-b87d-74f8f825457d from the persistent domain config.
Apr 14 04:23:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance a0a20aeb-ed06-4c65-b87d-74f8f825457d from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:23:32 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:23:32 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:23:32 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:23:32 user nova-compute[71317]:   <serial>3e0082ad-1ad0-4980-a066-6017f65fce9e</serial>
Apr 14 04:23:32 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 04:23:32 user nova-compute[71317]: </disk>
Apr 14 04:23:32 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.173s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:32 user nova-compute[71317]: DEBUG nova.network.neutron [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Successfully created port: d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.229s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446213.4368424, a0a20aeb-ed06-4c65-b87d-74f8f825457d => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:23:33 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully detached device vdb from instance a0a20aeb-ed06-4c65-b87d-74f8f825457d from the live domain config.
Apr 14 04:23:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YjWqRKtu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:23:33 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.058s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[496e90d1-03b7-4ee6-8e7a-3f0a47c3c309]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:23:33 user nova-compute[71317]: No targets found.
Apr 14 04:23:33 user nova-compute[71317]: iSNS:
Apr 14 04:23:33 user nova-compute[71317]: No targets found.
Apr 14 04:23:33 user nova-compute[71317]: STATIC:
Apr 14 04:23:33 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e
Apr 14 04:23:33 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:23:33 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:23:33 user nova-compute[71317]: FIRMWARE:
Apr 14 04:23:33 user nova-compute[71317]: No targets found.
Apr 14 04:23:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:23:33 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:23:33 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ede3929-4d90-4f22-a613-ddee29cc4775]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd0509b8-6f4f-4e25-9daa-37f409c12e1f]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:23:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e (non-flash)
Apr 14 04:23:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c41ffa30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e'): ({'sdf'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Removing single pathed devices sdf {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.243s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.035s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[abf5d199-0c15-4ed9-aa34-ab6971797a42]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0649d729-fbb7-471b-a0ce-32f81b577ec9]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flushing IO for device /dev/sdf {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f1f5930-fbf6-4f0b-893a-9f64b3959b04]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.056s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f7a0f5e-a9c5-4a45-95ad-40f386dfec45]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[376a0e4d-9786-46c7-b6bf-8ba75f2299ed]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:23:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --logout" returned: 0 in 0.084s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e37db2b-7192-4640-aee8-af82f930c299]: (4, ('Logging out of session [sid: 20, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260]\nLogout of [sid: 20, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 20, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260]
Apr 14 04:23:34 user nova-compute[71317]: Logout of [sid: 20, target: iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e, portal: 172.16.0.220,3260] successful.
Apr 14 04:23:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e0082ad-1ad0-4980-a066-6017f65fce9e -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fc95481-fc5f-49e9-bb18-6a807a022ee7]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.566s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== disconnect_volume: return (569ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.224s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json" returned: 0 in 0.210s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG nova.network.neutron [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Successfully updated port: d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "refresh_cache-7ffb8a8e-b24f-46e8-84c4-130e9f18cf55" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquired lock "refresh_cache-7ffb8a8e-b24f-46e8-84c4-130e9f18cf55" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG nova.network.neutron [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG nova.compute.manager [req-3dfde0ff-8b7a-4ffd-a067-30ab801a05b4 req-c5929d07-0c3d-435f-b25c-608565bac72f service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Received event network-changed-d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG nova.compute.manager [req-3dfde0ff-8b7a-4ffd-a067-30ab801a05b4 req-c5929d07-0c3d-435f-b25c-608565bac72f service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Refreshing instance network info cache due to event network-changed-d2a30aa1-7a43-4150-97c4-f312dea34ae7. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3dfde0ff-8b7a-4ffd-a067-30ab801a05b4 req-c5929d07-0c3d-435f-b25c-608565bac72f service nova] Acquiring lock "refresh_cache-7ffb8a8e-b24f-46e8-84c4-130e9f18cf55" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG nova.network.neutron [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.objects.instance [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'flavor' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9c6ca001-9179-4ec1-86a8-84541ed41c44 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.712s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.network.neutron [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Updating instance_info_cache with network_info: [{"id": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "address": "fa:16:3e:4b:ba:b4", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd2a30aa1-7a", "ovs_interfaceid": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Releasing lock "refresh_cache-7ffb8a8e-b24f-46e8-84c4-130e9f18cf55" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Instance network_info: |[{"id": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "address": "fa:16:3e:4b:ba:b4", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd2a30aa1-7a", "ovs_interfaceid": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3dfde0ff-8b7a-4ffd-a067-30ab801a05b4 req-c5929d07-0c3d-435f-b25c-608565bac72f service nova] Acquired lock "refresh_cache-7ffb8a8e-b24f-46e8-84c4-130e9f18cf55" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.network.neutron [req-3dfde0ff-8b7a-4ffd-a067-30ab801a05b4 req-c5929d07-0c3d-435f-b25c-608565bac72f service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Refreshing network info cache for port d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Start _get_guest_xml network_info=[{"id": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "address": "fa:16:3e:4b:ba:b4", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd2a30aa1-7a", "ovs_interfaceid": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:23:35 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:23:35 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:23:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1021322658',display_name='tempest-VolumesAdminNegativeTest-server-1021322658',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1021322658',id=21,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3d389bb78d2e40b9b07d2d3ecdb7566b',ramdisk_id='',reservation_id='r-d1yz9bco',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-618964915',owner_user_name='tempest-VolumesAdminNegativeTest-618964915-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:23:32Z,user_data=None,user_id='d2561bbc0c6d447da333ffdbb9fdc5f4',uuid=7ffb8a8e-b24f-46e8-84c4-130e9f18cf55,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "address": "fa:16:3e:4b:ba:b4", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd2a30aa1-7a", "ovs_interfaceid": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Converting VIF {"id": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "address": "fa:16:3e:4b:ba:b4", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd2a30aa1-7a", "ovs_interfaceid": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:4b:ba:b4,bridge_name='br-int',has_traffic_filtering=True,id=d2a30aa1-7a43-4150-97c4-f312dea34ae7,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd2a30aa1-7a') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.objects.instance [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lazy-loading 'pci_devices' on Instance uuid 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.207s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:23:35 user nova-compute[71317]:   <uuid>7ffb8a8e-b24f-46e8-84c4-130e9f18cf55</uuid>
Apr 14 04:23:35 user nova-compute[71317]:   <name>instance-00000015</name>
Apr 14 04:23:35 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:23:35 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:23:35 user nova-compute[71317]:   <metadata>
Apr 14 04:23:35 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:23:35 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:23:35 user nova-compute[71317]:       <nova:name>tempest-VolumesAdminNegativeTest-server-1021322658</nova:name>
Apr 14 04:23:35 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:23:35</nova:creationTime>
Apr 14 04:23:35 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:23:35 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:23:35 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:23:35 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:23:35 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:23:35 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:23:35 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:23:35 user nova-compute[71317]:       <nova:owner>
Apr 14 04:23:35 user nova-compute[71317]:         <nova:user uuid="d2561bbc0c6d447da333ffdbb9fdc5f4">tempest-VolumesAdminNegativeTest-618964915-project-member</nova:user>
Apr 14 04:23:35 user nova-compute[71317]:         <nova:project uuid="3d389bb78d2e40b9b07d2d3ecdb7566b">tempest-VolumesAdminNegativeTest-618964915</nova:project>
Apr 14 04:23:35 user nova-compute[71317]:       </nova:owner>
Apr 14 04:23:35 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:23:35 user nova-compute[71317]:       <nova:ports>
Apr 14 04:23:35 user nova-compute[71317]:         <nova:port uuid="d2a30aa1-7a43-4150-97c4-f312dea34ae7">
Apr 14 04:23:35 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 04:23:35 user nova-compute[71317]:         </nova:port>
Apr 14 04:23:35 user nova-compute[71317]:       </nova:ports>
Apr 14 04:23:35 user nova-compute[71317]:     </nova:instance>
Apr 14 04:23:35 user nova-compute[71317]:   </metadata>
Apr 14 04:23:35 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:23:35 user nova-compute[71317]:     <system>
Apr 14 04:23:35 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:23:35 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:23:35 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:23:35 user nova-compute[71317]:       <entry name="serial">7ffb8a8e-b24f-46e8-84c4-130e9f18cf55</entry>
Apr 14 04:23:35 user nova-compute[71317]:       <entry name="uuid">7ffb8a8e-b24f-46e8-84c4-130e9f18cf55</entry>
Apr 14 04:23:35 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:23:35 user nova-compute[71317]:     </system>
Apr 14 04:23:35 user nova-compute[71317]:   </sysinfo>
Apr 14 04:23:35 user nova-compute[71317]:   <os>
Apr 14 04:23:35 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:23:35 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:23:35 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:23:35 user nova-compute[71317]:   </os>
Apr 14 04:23:35 user nova-compute[71317]:   <features>
Apr 14 04:23:35 user nova-compute[71317]:     <acpi/>
Apr 14 04:23:35 user nova-compute[71317]:     <apic/>
Apr 14 04:23:35 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:23:35 user nova-compute[71317]:   </features>
Apr 14 04:23:35 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:23:35 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:23:35 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:23:35 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:23:35 user nova-compute[71317]:   </clock>
Apr 14 04:23:35 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:23:35 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:23:35 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:23:35 user nova-compute[71317]:   </cpu>
Apr 14 04:23:35 user nova-compute[71317]:   <devices>
Apr 14 04:23:35 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:23:35 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:23:35 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk"/>
Apr 14 04:23:35 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:23:35 user nova-compute[71317]:     </disk>
Apr 14 04:23:35 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:23:35 user nova-compute[71317]:       <mac address="fa:16:3e:4b:ba:b4"/>
Apr 14 04:23:35 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:23:35 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:23:35 user nova-compute[71317]:       <target dev="tapd2a30aa1-7a"/>
Apr 14 04:23:35 user nova-compute[71317]:     </interface>
Apr 14 04:23:35 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:23:35 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/console.log" append="off"/>
Apr 14 04:23:35 user nova-compute[71317]:     </serial>
Apr 14 04:23:35 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:23:35 user nova-compute[71317]:     <video>
Apr 14 04:23:35 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:23:35 user nova-compute[71317]:     </video>
Apr 14 04:23:35 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:23:35 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:23:35 user nova-compute[71317]:     </rng>
Apr 14 04:23:35 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:23:35 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:23:35 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:23:35 user nova-compute[71317]:     </memballoon>
Apr 14 04:23:35 user nova-compute[71317]:   </devices>
Apr 14 04:23:35 user nova-compute[71317]: </domain>
Apr 14 04:23:35 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:23:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1021322658',display_name='tempest-VolumesAdminNegativeTest-server-1021322658',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1021322658',id=21,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3d389bb78d2e40b9b07d2d3ecdb7566b',ramdisk_id='',reservation_id='r-d1yz9bco',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-618964915',owner_user_name='tempest-VolumesAdminNegativeTest-618964915-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:23:32Z,user_data=None,user_id='d2561bbc0c6d447da333ffdbb9fdc5f4',uuid=7ffb8a8e-b24f-46e8-84c4-130e9f18cf55,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "address": "fa:16:3e:4b:ba:b4", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd2a30aa1-7a", "ovs_interfaceid": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Converting VIF {"id": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "address": "fa:16:3e:4b:ba:b4", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd2a30aa1-7a", "ovs_interfaceid": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:4b:ba:b4,bridge_name='br-int',has_traffic_filtering=True,id=d2a30aa1-7a43-4150-97c4-f312dea34ae7,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd2a30aa1-7a') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG os_vif [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:4b:ba:b4,bridge_name='br-int',has_traffic_filtering=True,id=d2a30aa1-7a43-4150-97c4-f312dea34ae7,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd2a30aa1-7a') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapd2a30aa1-7a, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapd2a30aa1-7a, col_values=(('external_ids', {'iface-id': 'd2a30aa1-7a43-4150-97c4-f312dea34ae7', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:4b:ba:b4', 'vm-uuid': '7ffb8a8e-b24f-46e8-84c4-130e9f18cf55'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:35 user nova-compute[71317]: INFO os_vif [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:4b:ba:b4,bridge_name='br-int',has_traffic_filtering=True,id=d2a30aa1-7a43-4150-97c4-f312dea34ae7,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd2a30aa1-7a')
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] No VIF found with MAC fa:16:3e:4b:ba:b4, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:36 user nova-compute[71317]: DEBUG nova.network.neutron [req-3dfde0ff-8b7a-4ffd-a067-30ab801a05b4 req-c5929d07-0c3d-435f-b25c-608565bac72f service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Updated VIF entry in instance network info cache for port d2a30aa1-7a43-4150-97c4-f312dea34ae7. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:23:36 user nova-compute[71317]: DEBUG nova.network.neutron [req-3dfde0ff-8b7a-4ffd-a067-30ab801a05b4 req-c5929d07-0c3d-435f-b25c-608565bac72f service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Updating instance_info_cache with network_info: [{"id": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "address": "fa:16:3e:4b:ba:b4", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd2a30aa1-7a", "ovs_interfaceid": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:23:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3dfde0ff-8b7a-4ffd-a067-30ab801a05b4 req-c5929d07-0c3d-435f-b25c-608565bac72f service nova] Releasing lock "refresh_cache-7ffb8a8e-b24f-46e8-84c4-130e9f18cf55" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:23:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-1136504d-f554-41a3-8719-dd58fa9b170f req-f4c83fe4-3cc3-400c-b95c-58afb93cdd95 service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Received event network-vif-plugged-d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1136504d-f554-41a3-8719-dd58fa9b170f req-f4c83fe4-3cc3-400c-b95c-58afb93cdd95 service nova] Acquiring lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1136504d-f554-41a3-8719-dd58fa9b170f req-f4c83fe4-3cc3-400c-b95c-58afb93cdd95 service nova] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1136504d-f554-41a3-8719-dd58fa9b170f req-f4c83fe4-3cc3-400c-b95c-58afb93cdd95 service nova] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-1136504d-f554-41a3-8719-dd58fa9b170f req-f4c83fe4-3cc3-400c-b95c-58afb93cdd95 service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] No waiting events found dispatching network-vif-plugged-d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:23:37 user nova-compute[71317]: WARNING nova.compute.manager [req-1136504d-f554-41a3-8719-dd58fa9b170f req-f4c83fe4-3cc3-400c-b95c-58afb93cdd95 service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Received unexpected event network-vif-plugged-d2a30aa1-7a43-4150-97c4-f312dea34ae7 for instance with vm_state building and task_state spawning.
Apr 14 04:23:37 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:23:37 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7511MB free_disk=26.326004028320312GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 9b532a44-20b7-4590-babe-684c49598a62 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance da3b4651-aaa0-430d-85b8-c15de386d08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance d2e6cf32-080b-47b2-b4db-2b776ba85b38 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 97392a47-53f0-4bde-977a-e2945332fe1f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 46a5fd77-abec-4ab3-b519-7c649e63459c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1920MB phys_disk=40GB used_disk=11GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:23:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:38 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:23:38 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:23:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:23:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.605s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446219.4763901, 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:23:39 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] VM Resumed (Lifecycle Event)
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:23:39 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Instance spawned successfully.
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-178ebc30-119a-46c6-aece-28df7f68d437 req-f9b26940-9ffa-427e-b45b-c253b99fe61b service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Received event network-vif-plugged-d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-178ebc30-119a-46c6-aece-28df7f68d437 req-f9b26940-9ffa-427e-b45b-c253b99fe61b service nova] Acquiring lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-178ebc30-119a-46c6-aece-28df7f68d437 req-f9b26940-9ffa-427e-b45b-c253b99fe61b service nova] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-178ebc30-119a-46c6-aece-28df7f68d437 req-f9b26940-9ffa-427e-b45b-c253b99fe61b service nova] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-178ebc30-119a-46c6-aece-28df7f68d437 req-f9b26940-9ffa-427e-b45b-c253b99fe61b service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] No waiting events found dispatching network-vif-plugged-d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:23:39 user nova-compute[71317]: WARNING nova.compute.manager [req-178ebc30-119a-46c6-aece-28df7f68d437 req-f9b26940-9ffa-427e-b45b-c253b99fe61b service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Received unexpected event network-vif-plugged-d2a30aa1-7a43-4150-97c4-f312dea34ae7 for instance with vm_state building and task_state spawning.
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:39 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446219.4798877, 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:23:39 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] VM Started (Lifecycle Event)
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:23:39 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:23:39 user nova-compute[71317]: INFO nova.compute.manager [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Took 7.92 seconds to spawn the instance on the hypervisor.
Apr 14 04:23:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:39 user nova-compute[71317]: INFO nova.compute.manager [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Took 8.71 seconds to build instance.
Apr 14 04:23:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-af3fa9ed-0e61-4d00-bfea-7a6a4f84380f tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.844s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:44 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:23:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:23:44 user nova-compute[71317]: INFO nova.compute.claims [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Claim successful on node user
Apr 14 04:23:45 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.541s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:23:45 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:23:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG nova.policy [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0d3c1b814ec240279646878fd090f5f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39731dbb51484eff890a74b07241ba75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:23:45 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Creating image(s)
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "/opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "/opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "/opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.164s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk 1073741824" returned: 0 in 0.057s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.211s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:23:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:46 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Cannot resize image /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:23:46 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'migration_context' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:23:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:23:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Ensure instance console log exists: /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:23:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:46 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Successfully created port: dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Successfully updated port: dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "refresh_cache-f593ceb0-de44-490b-b365-3a05a3acd066" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquired lock "refresh_cache-f593ceb0-de44-490b-b365-3a05a3acd066" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-fe5496f1-1ec3-4c35-99f7-e905cf65197b req-2737f551-64a6-4e20-b6e7-21a3151f5102 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Received event network-changed-dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-fe5496f1-1ec3-4c35-99f7-e905cf65197b req-2737f551-64a6-4e20-b6e7-21a3151f5102 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Refreshing instance network info cache due to event network-changed-dd12687e-ca27-4fba-9653-4fa1af3608a8. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-fe5496f1-1ec3-4c35-99f7-e905cf65197b req-2737f551-64a6-4e20-b6e7-21a3151f5102 service nova] Acquiring lock "refresh_cache-f593ceb0-de44-490b-b365-3a05a3acd066" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Updating instance_info_cache with network_info: [{"id": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "address": "fa:16:3e:17:59:61", "network": {"id": "694b7c0a-cb94-4b83-99aa-03c94c049086", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-383837474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "39731dbb51484eff890a74b07241ba75", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd12687e-ca", "ovs_interfaceid": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Releasing lock "refresh_cache-f593ceb0-de44-490b-b365-3a05a3acd066" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Instance network_info: |[{"id": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "address": "fa:16:3e:17:59:61", "network": {"id": "694b7c0a-cb94-4b83-99aa-03c94c049086", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-383837474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "39731dbb51484eff890a74b07241ba75", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd12687e-ca", "ovs_interfaceid": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-fe5496f1-1ec3-4c35-99f7-e905cf65197b req-2737f551-64a6-4e20-b6e7-21a3151f5102 service nova] Acquired lock "refresh_cache-f593ceb0-de44-490b-b365-3a05a3acd066" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.network.neutron [req-fe5496f1-1ec3-4c35-99f7-e905cf65197b req-2737f551-64a6-4e20-b6e7-21a3151f5102 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Refreshing network info cache for port dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Start _get_guest_xml network_info=[{"id": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "address": "fa:16:3e:17:59:61", "network": {"id": "694b7c0a-cb94-4b83-99aa-03c94c049086", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-383837474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "39731dbb51484eff890a74b07241ba75", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd12687e-ca", "ovs_interfaceid": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:23:47 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:23:47 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:23:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-141729612',display_name='tempest-SnapshotDataIntegrityTests-server-141729612',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-141729612',id=22,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOjIW2Ycb+mq2tNzF5oSfsdR8libq7W1H6v9ImKoXEDVO168nnN9ZHy7JgCycEV+p/J22r0BDP5EimQKsNhepd1EFrAa2ljn1iDdZ9Zn90UYL0Ok/xoK1pzqla/0GV9JdQ==',key_name='tempest-SnapshotDataIntegrityTests-881844326',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='39731dbb51484eff890a74b07241ba75',ramdisk_id='',reservation_id='r-kqgpo0fv',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-SnapshotDataIntegrityTests-502963555',owner_user_name='tempest-SnapshotDataIntegrityTests-502963555-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:23:45Z,user_data=None,user_id='0d3c1b814ec240279646878fd090f5f2',uuid=f593ceb0-de44-490b-b365-3a05a3acd066,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "address": "fa:16:3e:17:59:61", "network": {"id": "694b7c0a-cb94-4b83-99aa-03c94c049086", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-383837474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "39731dbb51484eff890a74b07241ba75", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd12687e-ca", "ovs_interfaceid": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Converting VIF {"id": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "address": "fa:16:3e:17:59:61", "network": {"id": "694b7c0a-cb94-4b83-99aa-03c94c049086", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-383837474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "39731dbb51484eff890a74b07241ba75", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd12687e-ca", "ovs_interfaceid": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:17:59:61,bridge_name='br-int',has_traffic_filtering=True,id=dd12687e-ca27-4fba-9653-4fa1af3608a8,network=Network(694b7c0a-cb94-4b83-99aa-03c94c049086),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd12687e-ca') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'pci_devices' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:23:47 user nova-compute[71317]:   <uuid>f593ceb0-de44-490b-b365-3a05a3acd066</uuid>
Apr 14 04:23:47 user nova-compute[71317]:   <name>instance-00000016</name>
Apr 14 04:23:47 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:23:47 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:23:47 user nova-compute[71317]:   <metadata>
Apr 14 04:23:47 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:23:47 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:23:47 user nova-compute[71317]:       <nova:name>tempest-SnapshotDataIntegrityTests-server-141729612</nova:name>
Apr 14 04:23:47 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:23:47</nova:creationTime>
Apr 14 04:23:47 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:23:47 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:23:47 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:23:47 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:23:47 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:23:47 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:23:47 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:23:47 user nova-compute[71317]:       <nova:owner>
Apr 14 04:23:47 user nova-compute[71317]:         <nova:user uuid="0d3c1b814ec240279646878fd090f5f2">tempest-SnapshotDataIntegrityTests-502963555-project-member</nova:user>
Apr 14 04:23:47 user nova-compute[71317]:         <nova:project uuid="39731dbb51484eff890a74b07241ba75">tempest-SnapshotDataIntegrityTests-502963555</nova:project>
Apr 14 04:23:47 user nova-compute[71317]:       </nova:owner>
Apr 14 04:23:47 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:23:47 user nova-compute[71317]:       <nova:ports>
Apr 14 04:23:47 user nova-compute[71317]:         <nova:port uuid="dd12687e-ca27-4fba-9653-4fa1af3608a8">
Apr 14 04:23:47 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 04:23:47 user nova-compute[71317]:         </nova:port>
Apr 14 04:23:47 user nova-compute[71317]:       </nova:ports>
Apr 14 04:23:47 user nova-compute[71317]:     </nova:instance>
Apr 14 04:23:47 user nova-compute[71317]:   </metadata>
Apr 14 04:23:47 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:23:47 user nova-compute[71317]:     <system>
Apr 14 04:23:47 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:23:47 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:23:47 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:23:47 user nova-compute[71317]:       <entry name="serial">f593ceb0-de44-490b-b365-3a05a3acd066</entry>
Apr 14 04:23:47 user nova-compute[71317]:       <entry name="uuid">f593ceb0-de44-490b-b365-3a05a3acd066</entry>
Apr 14 04:23:47 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:23:47 user nova-compute[71317]:     </system>
Apr 14 04:23:47 user nova-compute[71317]:   </sysinfo>
Apr 14 04:23:47 user nova-compute[71317]:   <os>
Apr 14 04:23:47 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:23:47 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:23:47 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:23:47 user nova-compute[71317]:   </os>
Apr 14 04:23:47 user nova-compute[71317]:   <features>
Apr 14 04:23:47 user nova-compute[71317]:     <acpi/>
Apr 14 04:23:47 user nova-compute[71317]:     <apic/>
Apr 14 04:23:47 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:23:47 user nova-compute[71317]:   </features>
Apr 14 04:23:47 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:23:47 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:23:47 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:23:47 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:23:47 user nova-compute[71317]:   </clock>
Apr 14 04:23:47 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:23:47 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:23:47 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:23:47 user nova-compute[71317]:   </cpu>
Apr 14 04:23:47 user nova-compute[71317]:   <devices>
Apr 14 04:23:47 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:23:47 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:23:47 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk"/>
Apr 14 04:23:47 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:23:47 user nova-compute[71317]:     </disk>
Apr 14 04:23:47 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:23:47 user nova-compute[71317]:       <mac address="fa:16:3e:17:59:61"/>
Apr 14 04:23:47 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:23:47 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:23:47 user nova-compute[71317]:       <target dev="tapdd12687e-ca"/>
Apr 14 04:23:47 user nova-compute[71317]:     </interface>
Apr 14 04:23:47 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:23:47 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/console.log" append="off"/>
Apr 14 04:23:47 user nova-compute[71317]:     </serial>
Apr 14 04:23:47 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:23:47 user nova-compute[71317]:     <video>
Apr 14 04:23:47 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:23:47 user nova-compute[71317]:     </video>
Apr 14 04:23:47 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:23:47 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:23:47 user nova-compute[71317]:     </rng>
Apr 14 04:23:47 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:23:47 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:23:47 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:23:47 user nova-compute[71317]:     </memballoon>
Apr 14 04:23:47 user nova-compute[71317]:   </devices>
Apr 14 04:23:47 user nova-compute[71317]: </domain>
Apr 14 04:23:47 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:23:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-141729612',display_name='tempest-SnapshotDataIntegrityTests-server-141729612',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-141729612',id=22,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOjIW2Ycb+mq2tNzF5oSfsdR8libq7W1H6v9ImKoXEDVO168nnN9ZHy7JgCycEV+p/J22r0BDP5EimQKsNhepd1EFrAa2ljn1iDdZ9Zn90UYL0Ok/xoK1pzqla/0GV9JdQ==',key_name='tempest-SnapshotDataIntegrityTests-881844326',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='39731dbb51484eff890a74b07241ba75',ramdisk_id='',reservation_id='r-kqgpo0fv',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-SnapshotDataIntegrityTests-502963555',owner_user_name='tempest-SnapshotDataIntegrityTests-502963555-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:23:45Z,user_data=None,user_id='0d3c1b814ec240279646878fd090f5f2',uuid=f593ceb0-de44-490b-b365-3a05a3acd066,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "address": "fa:16:3e:17:59:61", "network": {"id": "694b7c0a-cb94-4b83-99aa-03c94c049086", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-383837474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "39731dbb51484eff890a74b07241ba75", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd12687e-ca", "ovs_interfaceid": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Converting VIF {"id": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "address": "fa:16:3e:17:59:61", "network": {"id": "694b7c0a-cb94-4b83-99aa-03c94c049086", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-383837474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "39731dbb51484eff890a74b07241ba75", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd12687e-ca", "ovs_interfaceid": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:17:59:61,bridge_name='br-int',has_traffic_filtering=True,id=dd12687e-ca27-4fba-9653-4fa1af3608a8,network=Network(694b7c0a-cb94-4b83-99aa-03c94c049086),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd12687e-ca') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG os_vif [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:17:59:61,bridge_name='br-int',has_traffic_filtering=True,id=dd12687e-ca27-4fba-9653-4fa1af3608a8,network=Network(694b7c0a-cb94-4b83-99aa-03c94c049086),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd12687e-ca') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapdd12687e-ca, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapdd12687e-ca, col_values=(('external_ids', {'iface-id': 'dd12687e-ca27-4fba-9653-4fa1af3608a8', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:17:59:61', 'vm-uuid': 'f593ceb0-de44-490b-b365-3a05a3acd066'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:23:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:48 user nova-compute[71317]: INFO os_vif [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:17:59:61,bridge_name='br-int',has_traffic_filtering=True,id=dd12687e-ca27-4fba-9653-4fa1af3608a8,network=Network(694b7c0a-cb94-4b83-99aa-03c94c049086),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd12687e-ca')
Apr 14 04:23:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:23:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No VIF found with MAC fa:16:3e:17:59:61, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:23:48 user nova-compute[71317]: DEBUG nova.network.neutron [req-fe5496f1-1ec3-4c35-99f7-e905cf65197b req-2737f551-64a6-4e20-b6e7-21a3151f5102 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Updated VIF entry in instance network info cache for port dd12687e-ca27-4fba-9653-4fa1af3608a8. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:23:48 user nova-compute[71317]: DEBUG nova.network.neutron [req-fe5496f1-1ec3-4c35-99f7-e905cf65197b req-2737f551-64a6-4e20-b6e7-21a3151f5102 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Updating instance_info_cache with network_info: [{"id": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "address": "fa:16:3e:17:59:61", "network": {"id": "694b7c0a-cb94-4b83-99aa-03c94c049086", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-383837474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "39731dbb51484eff890a74b07241ba75", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd12687e-ca", "ovs_interfaceid": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:23:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-fe5496f1-1ec3-4c35-99f7-e905cf65197b req-2737f551-64a6-4e20-b6e7-21a3151f5102 service nova] Releasing lock "refresh_cache-f593ceb0-de44-490b-b365-3a05a3acd066" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:23:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-a5618da3-0b00-4aa4-acde-22f8ce83a326 req-5b01a2e4-5fe9-4fc4-8692-cde89bcfc159 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Received event network-vif-plugged-dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a5618da3-0b00-4aa4-acde-22f8ce83a326 req-5b01a2e4-5fe9-4fc4-8692-cde89bcfc159 service nova] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a5618da3-0b00-4aa4-acde-22f8ce83a326 req-5b01a2e4-5fe9-4fc4-8692-cde89bcfc159 service nova] Lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a5618da3-0b00-4aa4-acde-22f8ce83a326 req-5b01a2e4-5fe9-4fc4-8692-cde89bcfc159 service nova] Lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-a5618da3-0b00-4aa4-acde-22f8ce83a326 req-5b01a2e4-5fe9-4fc4-8692-cde89bcfc159 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] No waiting events found dispatching network-vif-plugged-dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:23:49 user nova-compute[71317]: WARNING nova.compute.manager [req-a5618da3-0b00-4aa4-acde-22f8ce83a326 req-5b01a2e4-5fe9-4fc4-8692-cde89bcfc159 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Received unexpected event network-vif-plugged-dd12687e-ca27-4fba-9653-4fa1af3608a8 for instance with vm_state building and task_state spawning.
Apr 14 04:23:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:23:51 user nova-compute[71317]: INFO nova.compute.claims [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Claim successful on node user
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446231.6702092, f593ceb0-de44-490b-b365-3a05a3acd066 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:23:51 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] VM Resumed (Lifecycle Event)
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:23:51 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Instance spawned successfully.
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:51 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446231.6722186, f593ceb0-de44-490b-b365-3a05a3acd066 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:23:51 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] VM Started (Lifecycle Event)
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:23:51 user nova-compute[71317]: INFO nova.compute.manager [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Took 6.39 seconds to spawn the instance on the hypervisor.
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:51 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.compute.manager [req-5387dbe8-b9e7-4e0c-b7be-a8e789c6e715 req-70875a6b-daaa-4f72-9385-73770e2b842d service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Received event network-vif-plugged-dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5387dbe8-b9e7-4e0c-b7be-a8e789c6e715 req-70875a6b-daaa-4f72-9385-73770e2b842d service nova] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5387dbe8-b9e7-4e0c-b7be-a8e789c6e715 req-70875a6b-daaa-4f72-9385-73770e2b842d service nova] Lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5387dbe8-b9e7-4e0c-b7be-a8e789c6e715 req-70875a6b-daaa-4f72-9385-73770e2b842d service nova] Lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.compute.manager [req-5387dbe8-b9e7-4e0c-b7be-a8e789c6e715 req-70875a6b-daaa-4f72-9385-73770e2b842d service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] No waiting events found dispatching network-vif-plugged-dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:23:51 user nova-compute[71317]: WARNING nova.compute.manager [req-5387dbe8-b9e7-4e0c-b7be-a8e789c6e715 req-70875a6b-daaa-4f72-9385-73770e2b842d service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Received unexpected event network-vif-plugged-dd12687e-ca27-4fba-9653-4fa1af3608a8 for instance with vm_state building and task_state spawning.
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:23:51 user nova-compute[71317]: INFO nova.compute.manager [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Took 7.34 seconds to build instance.
Apr 14 04:23:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.597s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:23:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a700cdcf-0e40-460b-bd7d-3166757cc86f tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.447s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:23:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:23:52 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:23:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:23:52 user nova-compute[71317]: DEBUG nova.policy [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '982f167333954612a51911fc8d6c1ce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a60f60717b44f4ba904a949eabfe608', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:23:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:23:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:23:52 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Creating image(s)
Apr 14 04:23:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "/opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "/opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "/opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "76dfe33fcdd199f313fccbe5149d862d0c92a592" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "76dfe33fcdd199f313fccbe5149d862d0c92a592" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592.part --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592.part --force-share --output=json" returned: 0 in 0.185s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG nova.virt.images [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] 0b860afe-1945-4661-a55b-1a6ae0f644ea was qcow2, converting to raw {{(pid=71317) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592.part /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592.converted {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Successfully created port: dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592.part /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592.converted" returned: 0 in 0.297s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592.converted --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592.converted --force-share --output=json" returned: 0 in 0.153s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "76dfe33fcdd199f313fccbe5149d862d0c92a592" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.532s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592 --force-share --output=json" returned: 0 in 0.150s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "76dfe33fcdd199f313fccbe5149d862d0c92a592" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "76dfe33fcdd199f313fccbe5149d862d0c92a592" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592 --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592,backing_fmt=raw /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592,backing_fmt=raw /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk 1073741824" returned: 0 in 0.051s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "76dfe33fcdd199f313fccbe5149d862d0c92a592" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.202s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Successfully updated port: dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquired lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-f963567f-b626-4a06-9d47-910c566c2721 req-753d9ef6-72d4-4ee8-aec2-391332914215 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-changed-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-f963567f-b626-4a06-9d47-910c566c2721 req-753d9ef6-72d4-4ee8-aec2-391332914215 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Refreshing instance network info cache due to event network-changed-dc662cd5-f74a-40ee-bc14-e0948cbc4f36. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f963567f-b626-4a06-9d47-910c566c2721 req-753d9ef6-72d4-4ee8-aec2-391332914215 service nova] Acquiring lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592 --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Checking if we can resize image /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Cannot resize image /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'migration_context' on Instance uuid 5273d202-ffc5-43c6-bb4f-f6d1a6855271 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Ensure instance console log exists: /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updating instance_info_cache with network_info: [{"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Releasing lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Instance network_info: |[{"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f963567f-b626-4a06-9d47-910c566c2721 req-753d9ef6-72d4-4ee8-aec2-391332914215 service nova] Acquired lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-f963567f-b626-4a06-9d47-910c566c2721 req-753d9ef6-72d4-4ee8-aec2-391332914215 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Refreshing network info cache for port dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Start _get_guest_xml network_info=[{"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:23:48Z,direct_url=<?>,disk_format='qcow2',id=0b860afe-1945-4661-a55b-1a6ae0f644ea,min_disk=0,min_ram=0,name='tempest-scenario-img--2094177030',owner='0a60f60717b44f4ba904a949eabfe608',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:23:50Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '0b860afe-1945-4661-a55b-1a6ae0f644ea'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:23:54 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:23:54 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:23:48Z,direct_url=<?>,disk_format='qcow2',id=0b860afe-1945-4661-a55b-1a6ae0f644ea,min_disk=0,min_ram=0,name='tempest-scenario-img--2094177030',owner='0a60f60717b44f4ba904a949eabfe608',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:23:50Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:23:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-593822043',display_name='tempest-TestMinimumBasicScenario-server-593822043',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-593822043',id=23,image_ref='0b860afe-1945-4661-a55b-1a6ae0f644ea',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLyXBaFsreuBQDAM6pDqgr0rCR6Xh6YpqhuY/rJBXhVu6O6Jb6D2jMCp0EWMBknFKHHS7zo9OjEuZQRy7TrI+gfDrzjImHlLab3mcGep976OxV4nuHSlwuq9G12jktVdmA==',key_name='tempest-TestMinimumBasicScenario-231821443',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0a60f60717b44f4ba904a949eabfe608',ramdisk_id='',reservation_id='r-sji3zlhc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='0b860afe-1945-4661-a55b-1a6ae0f644ea',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1872250892',owner_user_name='tempest-TestMinimumBasicScenario-1872250892-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:23:52Z,user_data=None,user_id='982f167333954612a51911fc8d6c1ce9',uuid=5273d202-ffc5-43c6-bb4f-f6d1a6855271,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converting VIF {"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:27:5d:35,bridge_name='br-int',has_traffic_filtering=True,id=dc662cd5-f74a-40ee-bc14-e0948cbc4f36,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc662cd5-f7') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'pci_devices' on Instance uuid 5273d202-ffc5-43c6-bb4f-f6d1a6855271 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:23:54 user nova-compute[71317]:   <uuid>5273d202-ffc5-43c6-bb4f-f6d1a6855271</uuid>
Apr 14 04:23:54 user nova-compute[71317]:   <name>instance-00000017</name>
Apr 14 04:23:54 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:23:54 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:23:54 user nova-compute[71317]:   <metadata>
Apr 14 04:23:54 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:23:54 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:23:54 user nova-compute[71317]:       <nova:name>tempest-TestMinimumBasicScenario-server-593822043</nova:name>
Apr 14 04:23:54 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:23:54</nova:creationTime>
Apr 14 04:23:54 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:23:54 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:23:54 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:23:54 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:23:54 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:23:54 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:23:54 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:23:54 user nova-compute[71317]:       <nova:owner>
Apr 14 04:23:54 user nova-compute[71317]:         <nova:user uuid="982f167333954612a51911fc8d6c1ce9">tempest-TestMinimumBasicScenario-1872250892-project-member</nova:user>
Apr 14 04:23:54 user nova-compute[71317]:         <nova:project uuid="0a60f60717b44f4ba904a949eabfe608">tempest-TestMinimumBasicScenario-1872250892</nova:project>
Apr 14 04:23:54 user nova-compute[71317]:       </nova:owner>
Apr 14 04:23:54 user nova-compute[71317]:       <nova:root type="image" uuid="0b860afe-1945-4661-a55b-1a6ae0f644ea"/>
Apr 14 04:23:54 user nova-compute[71317]:       <nova:ports>
Apr 14 04:23:54 user nova-compute[71317]:         <nova:port uuid="dc662cd5-f74a-40ee-bc14-e0948cbc4f36">
Apr 14 04:23:54 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Apr 14 04:23:54 user nova-compute[71317]:         </nova:port>
Apr 14 04:23:54 user nova-compute[71317]:       </nova:ports>
Apr 14 04:23:54 user nova-compute[71317]:     </nova:instance>
Apr 14 04:23:54 user nova-compute[71317]:   </metadata>
Apr 14 04:23:54 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:23:54 user nova-compute[71317]:     <system>
Apr 14 04:23:54 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:23:54 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:23:54 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:23:54 user nova-compute[71317]:       <entry name="serial">5273d202-ffc5-43c6-bb4f-f6d1a6855271</entry>
Apr 14 04:23:54 user nova-compute[71317]:       <entry name="uuid">5273d202-ffc5-43c6-bb4f-f6d1a6855271</entry>
Apr 14 04:23:54 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:23:54 user nova-compute[71317]:     </system>
Apr 14 04:23:54 user nova-compute[71317]:   </sysinfo>
Apr 14 04:23:54 user nova-compute[71317]:   <os>
Apr 14 04:23:54 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:23:54 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:23:54 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:23:54 user nova-compute[71317]:   </os>
Apr 14 04:23:54 user nova-compute[71317]:   <features>
Apr 14 04:23:54 user nova-compute[71317]:     <acpi/>
Apr 14 04:23:54 user nova-compute[71317]:     <apic/>
Apr 14 04:23:54 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:23:54 user nova-compute[71317]:   </features>
Apr 14 04:23:54 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:23:54 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:23:54 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:23:54 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:23:54 user nova-compute[71317]:   </clock>
Apr 14 04:23:54 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:23:54 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:23:54 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:23:54 user nova-compute[71317]:   </cpu>
Apr 14 04:23:54 user nova-compute[71317]:   <devices>
Apr 14 04:23:54 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:23:54 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:23:54 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk"/>
Apr 14 04:23:54 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:23:54 user nova-compute[71317]:     </disk>
Apr 14 04:23:54 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:23:54 user nova-compute[71317]:       <mac address="fa:16:3e:27:5d:35"/>
Apr 14 04:23:54 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:23:54 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:23:54 user nova-compute[71317]:       <target dev="tapdc662cd5-f7"/>
Apr 14 04:23:54 user nova-compute[71317]:     </interface>
Apr 14 04:23:54 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:23:54 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/console.log" append="off"/>
Apr 14 04:23:54 user nova-compute[71317]:     </serial>
Apr 14 04:23:54 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:23:54 user nova-compute[71317]:     <video>
Apr 14 04:23:54 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:23:54 user nova-compute[71317]:     </video>
Apr 14 04:23:54 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:23:54 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:23:54 user nova-compute[71317]:     </rng>
Apr 14 04:23:54 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:23:54 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:23:54 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:23:54 user nova-compute[71317]:     </memballoon>
Apr 14 04:23:54 user nova-compute[71317]:   </devices>
Apr 14 04:23:54 user nova-compute[71317]: </domain>
Apr 14 04:23:54 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:23:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-593822043',display_name='tempest-TestMinimumBasicScenario-server-593822043',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-593822043',id=23,image_ref='0b860afe-1945-4661-a55b-1a6ae0f644ea',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLyXBaFsreuBQDAM6pDqgr0rCR6Xh6YpqhuY/rJBXhVu6O6Jb6D2jMCp0EWMBknFKHHS7zo9OjEuZQRy7TrI+gfDrzjImHlLab3mcGep976OxV4nuHSlwuq9G12jktVdmA==',key_name='tempest-TestMinimumBasicScenario-231821443',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0a60f60717b44f4ba904a949eabfe608',ramdisk_id='',reservation_id='r-sji3zlhc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='0b860afe-1945-4661-a55b-1a6ae0f644ea',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1872250892',owner_user_name='tempest-TestMinimumBasicScenario-1872250892-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:23:52Z,user_data=None,user_id='982f167333954612a51911fc8d6c1ce9',uuid=5273d202-ffc5-43c6-bb4f-f6d1a6855271,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converting VIF {"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:27:5d:35,bridge_name='br-int',has_traffic_filtering=True,id=dc662cd5-f74a-40ee-bc14-e0948cbc4f36,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc662cd5-f7') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG os_vif [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:27:5d:35,bridge_name='br-int',has_traffic_filtering=True,id=dc662cd5-f74a-40ee-bc14-e0948cbc4f36,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc662cd5-f7') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapdc662cd5-f7, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapdc662cd5-f7, col_values=(('external_ids', {'iface-id': 'dc662cd5-f74a-40ee-bc14-e0948cbc4f36', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:27:5d:35', 'vm-uuid': '5273d202-ffc5-43c6-bb4f-f6d1a6855271'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:54 user nova-compute[71317]: INFO os_vif [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:27:5d:35,bridge_name='br-int',has_traffic_filtering=True,id=dc662cd5-f74a-40ee-bc14-e0948cbc4f36,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc662cd5-f7')
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:23:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] No VIF found with MAC fa:16:3e:27:5d:35, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:23:55 user nova-compute[71317]: DEBUG nova.network.neutron [req-f963567f-b626-4a06-9d47-910c566c2721 req-753d9ef6-72d4-4ee8-aec2-391332914215 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updated VIF entry in instance network info cache for port dc662cd5-f74a-40ee-bc14-e0948cbc4f36. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:23:55 user nova-compute[71317]: DEBUG nova.network.neutron [req-f963567f-b626-4a06-9d47-910c566c2721 req-753d9ef6-72d4-4ee8-aec2-391332914215 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updating instance_info_cache with network_info: [{"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:23:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f963567f-b626-4a06-9d47-910c566c2721 req-753d9ef6-72d4-4ee8-aec2-391332914215 service nova] Releasing lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-8993cfa1-fd42-4859-b76a-9604307e5608 req-525053ec-abc6-45c5-987a-ab1a0879dfb0 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8993cfa1-fd42-4859-b76a-9604307e5608 req-525053ec-abc6-45c5-987a-ab1a0879dfb0 service nova] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8993cfa1-fd42-4859-b76a-9604307e5608 req-525053ec-abc6-45c5-987a-ab1a0879dfb0 service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8993cfa1-fd42-4859-b76a-9604307e5608 req-525053ec-abc6-45c5-987a-ab1a0879dfb0 service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-8993cfa1-fd42-4859-b76a-9604307e5608 req-525053ec-abc6-45c5-987a-ab1a0879dfb0 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] No waiting events found dispatching network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:23:56 user nova-compute[71317]: WARNING nova.compute.manager [req-8993cfa1-fd42-4859-b76a-9604307e5608 req-525053ec-abc6-45c5-987a-ab1a0879dfb0 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received unexpected event network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 for instance with vm_state building and task_state spawning.
Apr 14 04:23:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446238.4643393, 5273d202-ffc5-43c6-bb4f-f6d1a6855271 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:23:58 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] VM Resumed (Lifecycle Event)
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:23:58 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Instance spawned successfully.
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-f23e63c8-6fd5-49ee-9278-b8e927086e46 req-d8d15639-24b5-4d15-9bb4-5565128e95f1 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f23e63c8-6fd5-49ee-9278-b8e927086e46 req-d8d15639-24b5-4d15-9bb4-5565128e95f1 service nova] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f23e63c8-6fd5-49ee-9278-b8e927086e46 req-d8d15639-24b5-4d15-9bb4-5565128e95f1 service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f23e63c8-6fd5-49ee-9278-b8e927086e46 req-d8d15639-24b5-4d15-9bb4-5565128e95f1 service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-f23e63c8-6fd5-49ee-9278-b8e927086e46 req-d8d15639-24b5-4d15-9bb4-5565128e95f1 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] No waiting events found dispatching network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:23:58 user nova-compute[71317]: WARNING nova.compute.manager [req-f23e63c8-6fd5-49ee-9278-b8e927086e46 req-d8d15639-24b5-4d15-9bb4-5565128e95f1 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received unexpected event network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 for instance with vm_state building and task_state spawning.
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:23:58 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446238.4680717, 5273d202-ffc5-43c6-bb4f-f6d1a6855271 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:23:58 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] VM Started (Lifecycle Event)
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:23:58 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:23:58 user nova-compute[71317]: INFO nova.compute.manager [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Took 6.36 seconds to spawn the instance on the hypervisor.
Apr 14 04:23:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:23:58 user nova-compute[71317]: INFO nova.compute.manager [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Took 7.30 seconds to build instance.
Apr 14 04:23:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c8b5d40f-9dcd-4c55-b76b-3028eaf6907e tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.394s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:23:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:05 user nova-compute[71317]: INFO nova.compute.manager [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Terminating instance
Apr 14 04:24:05 user nova-compute[71317]: DEBUG nova.compute.manager [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG nova.compute.manager [req-f3de1a0c-990f-4ee9-b96b-914b1e3a5437 req-81d7a2df-040e-47a1-a38f-789b980d0659 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-unplugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f3de1a0c-990f-4ee9-b96b-914b1e3a5437 req-81d7a2df-040e-47a1-a38f-789b980d0659 service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f3de1a0c-990f-4ee9-b96b-914b1e3a5437 req-81d7a2df-040e-47a1-a38f-789b980d0659 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f3de1a0c-990f-4ee9-b96b-914b1e3a5437 req-81d7a2df-040e-47a1-a38f-789b980d0659 service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-f3de1a0c-990f-4ee9-b96b-914b1e3a5437 req-81d7a2df-040e-47a1-a38f-789b980d0659 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-unplugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-f3de1a0c-990f-4ee9-b96b-914b1e3a5437 req-81d7a2df-040e-47a1-a38f-789b980d0659 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-unplugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:24:06 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Instance destroyed successfully.
Apr 14 04:24:06 user nova-compute[71317]: DEBUG nova.objects.instance [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lazy-loading 'resources' on Instance uuid cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-1335768391',display_name='tempest-ServerStableDeviceRescueTest-server-1335768391',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-1335768391',id=8,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHsERCJfV5vlojYp6CtYNDuskD1mPJwApYPpyopwqnFEg3bukLlc5MC0Mx5e6kcUmmhflLvGI9V/TTXxf//CKjD9nOwYO+5pKfSRGQB3deK57Z7HLLcWkH9v7l2Z+jP7Gg==',key_name='tempest-keypair-234703296',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:20:54Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='15bb9ddec81e4f6cbceb6ce7dee9c3a1',ramdisk_id='',reservation_id='r-9gfjdvk7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerStableDeviceRescueTest-1018370864',owner_user_name='tempest-ServerStableDeviceRescueTest-1018370864-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:22:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='676d80d3684045298d895555247174af',uuid=cd60eede-cd79-4c1b-adb7-ba7c6cbc9659,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Converting VIF {"id": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "address": "fa:16:3e:73:25:41", "network": {"id": "97d3cce1-27a3-4e76-9d3f-44860a2a8dcf", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1857887548-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "15bb9ddec81e4f6cbceb6ce7dee9c3a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0c62956f-a2", "ovs_interfaceid": "0c62956f-a2d8-4932-b48f-469eba99b8e6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:73:25:41,bridge_name='br-int',has_traffic_filtering=True,id=0c62956f-a2d8-4932-b48f-469eba99b8e6,network=Network(97d3cce1-27a3-4e76-9d3f-44860a2a8dcf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0c62956f-a2') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG os_vif [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:73:25:41,bridge_name='br-int',has_traffic_filtering=True,id=0c62956f-a2d8-4932-b48f-469eba99b8e6,network=Network(97d3cce1-27a3-4e76-9d3f-44860a2a8dcf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0c62956f-a2') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap0c62956f-a2, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:06 user nova-compute[71317]: INFO os_vif [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:73:25:41,bridge_name='br-int',has_traffic_filtering=True,id=0c62956f-a2d8-4932-b48f-469eba99b8e6,network=Network(97d3cce1-27a3-4e76-9d3f-44860a2a8dcf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0c62956f-a2')
Apr 14 04:24:06 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Deleting instance files /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659_del
Apr 14 04:24:06 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Deletion of /opt/stack/data/nova/instances/cd60eede-cd79-4c1b-adb7-ba7c6cbc9659_del complete
Apr 14 04:24:06 user nova-compute[71317]: INFO nova.compute.manager [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Took 0.90 seconds to destroy the instance on the hypervisor.
Apr 14 04:24:06 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:24:06 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:24:07 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:24:07 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Took 0.83 seconds to deallocate network for instance.
Apr 14 04:24:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-fbb2a2c2-d7b4-4f05-a4da-efd1124add7e req-a838604d-34ae-4f17-a188-cb6f69a8d094 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-deleted-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:07 user nova-compute[71317]: INFO nova.compute.manager [req-fbb2a2c2-d7b4-4f05-a4da-efd1124add7e req-a838604d-34ae-4f17-a188-cb6f69a8d094 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Neutron deleted interface 0c62956f-a2d8-4932-b48f-469eba99b8e6; detaching it from the instance and deleting it from the info cache
Apr 14 04:24:07 user nova-compute[71317]: DEBUG nova.network.neutron [req-fbb2a2c2-d7b4-4f05-a4da-efd1124add7e req-a838604d-34ae-4f17-a188-cb6f69a8d094 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:24:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-fbb2a2c2-d7b4-4f05-a4da-efd1124add7e req-a838604d-34ae-4f17-a188-cb6f69a8d094 service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Detach interface failed, port_id=0c62956f-a2d8-4932-b48f-469eba99b8e6, reason: Instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:24:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:07 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:24:07 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:24:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.394s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:07 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Deleted allocations for instance cd60eede-cd79-4c1b-adb7-ba7c6cbc9659
Apr 14 04:24:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b5441c7a-4783-4209-bba9-e60c55917de0 tempest-ServerStableDeviceRescueTest-1018370864 tempest-ServerStableDeviceRescueTest-1018370864-project-member] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.316s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-90f48607-950c-4252-ad5b-78a63f13dac3 req-d80de21a-cd11-4d48-88dc-b5289fafecef service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-90f48607-950c-4252-ad5b-78a63f13dac3 req-d80de21a-cd11-4d48-88dc-b5289fafecef service nova] Acquiring lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-90f48607-950c-4252-ad5b-78a63f13dac3 req-d80de21a-cd11-4d48-88dc-b5289fafecef service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-90f48607-950c-4252-ad5b-78a63f13dac3 req-d80de21a-cd11-4d48-88dc-b5289fafecef service nova] Lock "cd60eede-cd79-4c1b-adb7-ba7c6cbc9659-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-90f48607-950c-4252-ad5b-78a63f13dac3 req-d80de21a-cd11-4d48-88dc-b5289fafecef service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] No waiting events found dispatching network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:24:08 user nova-compute[71317]: WARNING nova.compute.manager [req-90f48607-950c-4252-ad5b-78a63f13dac3 req-d80de21a-cd11-4d48-88dc-b5289fafecef service nova] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Received unexpected event network-vif-plugged-0c62956f-a2d8-4932-b48f-469eba99b8e6 for instance with vm_state deleted and task_state None.
Apr 14 04:24:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:24:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquired lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:24:11 user nova-compute[71317]: DEBUG nova.network.neutron [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:24:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'info_cache' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.network.neutron [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updating instance_info_cache with network_info: [{"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Releasing lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:24:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance destroyed successfully.
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.objects.instance [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'resources' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1781752187',display_name='tempest-AttachVolumeTestJSON-server-1781752187',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1781752187',id=3,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBY2UT8M9EXLRm2waCk4jzyD2YisXrBUU8nZo6LT6g0AlR0LbKytQJ1iJL1X5k+1hK+8YQxwsn1kgdkATbgdoU6lyDlwBLIIaqdqxbWOahpeOBX2u8LLHQwm2K/x2XVeAg==',key_name='tempest-keypair-1428467741',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:16:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='6a89255339f74e55ad7c92967558bb81',ramdisk_id='',reservation_id='r-snsinryu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-991906288',owner_user_name='tempest-AttachVolumeTestJSON-991906288-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:22:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7610b45003e34f3395268977612f6c94',uuid=da3b4651-aaa0-430d-85b8-c15de386d08f,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converting VIF {"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG os_vif [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapdf63af46-27, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:24:12 user nova-compute[71317]: INFO os_vif [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27')
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Start _get_guest_xml network_info=[{"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.objects.instance [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'numa_topology' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:12 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:24:12 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.objects.instance [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'vcpu_model' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1781752187',display_name='tempest-AttachVolumeTestJSON-server-1781752187',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1781752187',id=3,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBY2UT8M9EXLRm2waCk4jzyD2YisXrBUU8nZo6LT6g0AlR0LbKytQJ1iJL1X5k+1hK+8YQxwsn1kgdkATbgdoU6lyDlwBLIIaqdqxbWOahpeOBX2u8LLHQwm2K/x2XVeAg==',key_name='tempest-keypair-1428467741',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:16:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='6a89255339f74e55ad7c92967558bb81',ramdisk_id='',reservation_id='r-snsinryu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-991906288',owner_user_name='tempest-AttachVolumeTestJSON-991906288-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:22:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7610b45003e34f3395268977612f6c94',uuid=da3b4651-aaa0-430d-85b8-c15de386d08f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converting VIF {"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.objects.instance [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'pci_devices' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:24:12 user nova-compute[71317]:   <uuid>da3b4651-aaa0-430d-85b8-c15de386d08f</uuid>
Apr 14 04:24:12 user nova-compute[71317]:   <name>instance-00000003</name>
Apr 14 04:24:12 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:24:12 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:24:12 user nova-compute[71317]:   <metadata>
Apr 14 04:24:12 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:24:12 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:24:12 user nova-compute[71317]:       <nova:name>tempest-AttachVolumeTestJSON-server-1781752187</nova:name>
Apr 14 04:24:12 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:24:12</nova:creationTime>
Apr 14 04:24:12 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:24:12 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:24:12 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:24:12 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:24:12 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:24:12 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:24:12 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:24:12 user nova-compute[71317]:       <nova:owner>
Apr 14 04:24:12 user nova-compute[71317]:         <nova:user uuid="7610b45003e34f3395268977612f6c94">tempest-AttachVolumeTestJSON-991906288-project-member</nova:user>
Apr 14 04:24:12 user nova-compute[71317]:         <nova:project uuid="6a89255339f74e55ad7c92967558bb81">tempest-AttachVolumeTestJSON-991906288</nova:project>
Apr 14 04:24:12 user nova-compute[71317]:       </nova:owner>
Apr 14 04:24:12 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:24:12 user nova-compute[71317]:       <nova:ports>
Apr 14 04:24:12 user nova-compute[71317]:         <nova:port uuid="df63af46-27d1-4086-ae2c-39e8bc829bc7">
Apr 14 04:24:12 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 04:24:12 user nova-compute[71317]:         </nova:port>
Apr 14 04:24:12 user nova-compute[71317]:       </nova:ports>
Apr 14 04:24:12 user nova-compute[71317]:     </nova:instance>
Apr 14 04:24:12 user nova-compute[71317]:   </metadata>
Apr 14 04:24:12 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:24:12 user nova-compute[71317]:     <system>
Apr 14 04:24:12 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:24:12 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:24:12 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:24:12 user nova-compute[71317]:       <entry name="serial">da3b4651-aaa0-430d-85b8-c15de386d08f</entry>
Apr 14 04:24:12 user nova-compute[71317]:       <entry name="uuid">da3b4651-aaa0-430d-85b8-c15de386d08f</entry>
Apr 14 04:24:12 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:24:12 user nova-compute[71317]:     </system>
Apr 14 04:24:12 user nova-compute[71317]:   </sysinfo>
Apr 14 04:24:12 user nova-compute[71317]:   <os>
Apr 14 04:24:12 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:24:12 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:24:12 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:24:12 user nova-compute[71317]:   </os>
Apr 14 04:24:12 user nova-compute[71317]:   <features>
Apr 14 04:24:12 user nova-compute[71317]:     <acpi/>
Apr 14 04:24:12 user nova-compute[71317]:     <apic/>
Apr 14 04:24:12 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:24:12 user nova-compute[71317]:   </features>
Apr 14 04:24:12 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:24:12 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:24:12 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:24:12 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:24:12 user nova-compute[71317]:   </clock>
Apr 14 04:24:12 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:24:12 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:24:12 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:24:12 user nova-compute[71317]:   </cpu>
Apr 14 04:24:12 user nova-compute[71317]:   <devices>
Apr 14 04:24:12 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:24:12 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:24:12 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk"/>
Apr 14 04:24:12 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:24:12 user nova-compute[71317]:     </disk>
Apr 14 04:24:12 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:24:12 user nova-compute[71317]:       <mac address="fa:16:3e:68:ca:2f"/>
Apr 14 04:24:12 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:24:12 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:24:12 user nova-compute[71317]:       <target dev="tapdf63af46-27"/>
Apr 14 04:24:12 user nova-compute[71317]:     </interface>
Apr 14 04:24:12 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:24:12 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/console.log" append="off"/>
Apr 14 04:24:12 user nova-compute[71317]:     </serial>
Apr 14 04:24:12 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:24:12 user nova-compute[71317]:     <video>
Apr 14 04:24:12 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:24:12 user nova-compute[71317]:     </video>
Apr 14 04:24:12 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:24:12 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:24:12 user nova-compute[71317]:     </rng>
Apr 14 04:24:12 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:24:12 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:24:12 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:24:12 user nova-compute[71317]:     </memballoon>
Apr 14 04:24:12 user nova-compute[71317]:   </devices>
Apr 14 04:24:12 user nova-compute[71317]: </domain>
Apr 14 04:24:12 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG nova.objects.instance [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'trusted_certs' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Checking if we can resize image /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Cannot resize image /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG nova.objects.instance [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'migration_context' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1781752187',display_name='tempest-AttachVolumeTestJSON-server-1781752187',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1781752187',id=3,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBY2UT8M9EXLRm2waCk4jzyD2YisXrBUU8nZo6LT6g0AlR0LbKytQJ1iJL1X5k+1hK+8YQxwsn1kgdkATbgdoU6lyDlwBLIIaqdqxbWOahpeOBX2u8LLHQwm2K/x2XVeAg==',key_name='tempest-keypair-1428467741',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:16:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=4,progress=0,project_id='6a89255339f74e55ad7c92967558bb81',ramdisk_id='',reservation_id='r-snsinryu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-991906288',owner_user_name='tempest-AttachVolumeTestJSON-991906288-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:22:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7610b45003e34f3395268977612f6c94',uuid=da3b4651-aaa0-430d-85b8-c15de386d08f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converting VIF {"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG os_vif [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapdf63af46-27, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapdf63af46-27, col_values=(('external_ids', {'iface-id': 'df63af46-27d1-4086-ae2c-39e8bc829bc7', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:68:ca:2f', 'vm-uuid': 'da3b4651-aaa0-430d-85b8-c15de386d08f'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:24:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:13 user nova-compute[71317]: INFO os_vif [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27')
Apr 14 04:24:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-195660a4-c391-4fb5-a88d-0f158742d383 req-f9680831-5e61-41d5-af25-fbb0e532285c service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-195660a4-c391-4fb5-a88d-0f158742d383 req-f9680831-5e61-41d5-af25-fbb0e532285c service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-195660a4-c391-4fb5-a88d-0f158742d383 req-f9680831-5e61-41d5-af25-fbb0e532285c service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-195660a4-c391-4fb5-a88d-0f158742d383 req-f9680831-5e61-41d5-af25-fbb0e532285c service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-195660a4-c391-4fb5-a88d-0f158742d383 req-f9680831-5e61-41d5-af25-fbb0e532285c service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:24:14 user nova-compute[71317]: WARNING nova.compute.manager [req-195660a4-c391-4fb5-a88d-0f158742d383 req-f9680831-5e61-41d5-af25-fbb0e532285c service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state stopped and task_state powering-on.
Apr 14 04:24:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:24:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:24:15 user nova-compute[71317]: INFO nova.compute.claims [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Claim successful on node user
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:24:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.489s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.network.neutron [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:24:16 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:24:16 user nova-compute[71317]: INFO nova.virt.block_device [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Booting with blank volume at /dev/vda
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.policy [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '6ec7103e06e643a6b9b2c908ba8d7b06', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd5995b934b0546c0983387e46e3b3df4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446256.8981876, da3b4651-aaa0-430d-85b8-c15de386d08f => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:24:16 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] VM Resumed (Lifecycle Event)
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:24:16 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance rebooted successfully.
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-807cc437-483e-4715-b3e1-0fe5923f965f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: stopped, current task_state: powering-on, current DB power_state: 4, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:24:16 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] During sync_power_state the instance has a pending task (powering-on). Skip.
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446256.9005663, da3b4651-aaa0-430d-85b8-c15de386d08f => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:24:16 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] VM Started (Lifecycle Event)
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:24:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:24:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-cbff7dd2-2e45-42fe-90c5-d81e04f5be6e req-fc11e447-2b41-40ef-9639-0744a8777273 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cbff7dd2-2e45-42fe-90c5-d81e04f5be6e req-fc11e447-2b41-40ef-9639-0744a8777273 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cbff7dd2-2e45-42fe-90c5-d81e04f5be6e req-fc11e447-2b41-40ef-9639-0744a8777273 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cbff7dd2-2e45-42fe-90c5-d81e04f5be6e req-fc11e447-2b41-40ef-9639-0744a8777273 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-cbff7dd2-2e45-42fe-90c5-d81e04f5be6e req-fc11e447-2b41-40ef-9639-0744a8777273 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:24:17 user nova-compute[71317]: WARNING nova.compute.manager [req-cbff7dd2-2e45-42fe-90c5-d81e04f5be6e req-fc11e447-2b41-40ef-9639-0744a8777273 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state active and task_state None.
Apr 14 04:24:17 user nova-compute[71317]: DEBUG nova.network.neutron [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Successfully created port: 6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:24:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Successfully updated port: 6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:24:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:24:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquired lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:24:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:24:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:24:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updating instance_info_cache with network_info: [{"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:24:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Releasing lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:24:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Instance network_info: |[{"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:24:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-07b64e28-0f60-466c-ae7f-42a73051d4cc req-dc493891-844a-49f1-b846-af7c33fa5b87 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-changed-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-07b64e28-0f60-466c-ae7f-42a73051d4cc req-dc493891-844a-49f1-b846-af7c33fa5b87 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Refreshing instance network info cache due to event network-changed-6baf0637-1ee9-4ccb-93a3-264041fb73f3. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:24:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-07b64e28-0f60-466c-ae7f-42a73051d4cc req-dc493891-844a-49f1-b846-af7c33fa5b87 service nova] Acquiring lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:24:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-07b64e28-0f60-466c-ae7f-42a73051d4cc req-dc493891-844a-49f1-b846-af7c33fa5b87 service nova] Acquired lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:24:19 user nova-compute[71317]: DEBUG nova.network.neutron [req-07b64e28-0f60-466c-ae7f-42a73051d4cc req-dc493891-844a-49f1-b846-af7c33fa5b87 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Refreshing network info cache for port 6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:24:19 user nova-compute[71317]: DEBUG nova.network.neutron [req-07b64e28-0f60-466c-ae7f-42a73051d4cc req-dc493891-844a-49f1-b846-af7c33fa5b87 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updated VIF entry in instance network info cache for port 6baf0637-1ee9-4ccb-93a3-264041fb73f3. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:24:19 user nova-compute[71317]: DEBUG nova.network.neutron [req-07b64e28-0f60-466c-ae7f-42a73051d4cc req-dc493891-844a-49f1-b846-af7c33fa5b87 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updating instance_info_cache with network_info: [{"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:24:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-07b64e28-0f60-466c-ae7f-42a73051d4cc req-dc493891-844a-49f1-b846-af7c33fa5b87 service nova] Releasing lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446246.408049, cd60eede-cd79-4c1b-adb7-ba7c6cbc9659 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:24:21 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] VM Stopped (Lifecycle Event)
Apr 14 04:24:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-f52bed0b-0e2f-4717-96f7-6fadaf87e726 None None] [instance: cd60eede-cd79-4c1b-adb7-ba7c6cbc9659] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG os_brick.utils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b38bd365-5d37-4f02-828c-ba30e0dfa1e7]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[68160e10-3f57-40eb-8633-97ae3904805b]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f663870-b26a-4726-9ae6-3fe61a9e80bb]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a7c0661-a36f-46ef-b038-0c164a3397dd]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:24:21 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:24:21 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a16d488b-e5a1-4079-83e2-74132d44f9f0]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG os_brick.utils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] <== get_connector_properties: return (117ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:24:21 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updating existing volume attachment record: 2931a803-f7ab-4f6c-b752-edcd2987b0d0 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:24:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.compute.manager [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:24:26 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Creating image(s)
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Ensure instance console log exists: /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Start _get_guest_xml network_info=[{"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YCyASkTZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '3df97db3-fd6c-4be6-af32-7cde9dbcce75', 'attached_at': '', 'detached_at': '', 'volume_id': '6e31b954-88d8-448b-bbb7-f19007a51866', 'serial': '6e31b954-88d8-448b-bbb7-f19007a51866'}, 'attachment_id': '2931a803-f7ab-4f6c-b752-edcd2987b0d0', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:24:26 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:24:26 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YCyASkTZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:24:26 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:24:26 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:24:26 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260" returned: 21 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[03af87a4-5c3b-486e-8440-4ca1ebc7a58a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:24:26 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d88e7f12-5a87-4af5-88f2-343a58564f9f]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866] added
Apr 14 04:24:26 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7c3300b-8c8e-40d4-b142-e493ee7186c9]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[949751d7-bc18-4b72-bb96-21b6b1e194cb]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YCyASkTZ {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YCyASkTZ" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ea2a325-5ce0-4efc-b5ba-f508babcb148]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YCyASkTZ'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[093a0261-7140-41ab-aa69-850eecfde01b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[76a20bdc-3c67-42a3-9792-e42074984fa2]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 14 04:24:26 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 14 04:24:26 user nova-compute[71317]:  {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:24:26 user nova-compute[71317]: WARNING os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 14 04:24:26 user nova-compute[71317]: 
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --login" returned: 0 in 0.053s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7af53da7-4025-4f46-ad90-797ac7835313]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866, portal: 172.16.0.220,3260]
Apr 14 04:24:26 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866, portal: 172.16.0.220,3260] successful.
Apr 14 04:24:26 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d435eea-e167-45a6-9150-234f6d81d96f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[94a394f9-c0f6-4c9e-95d0-3698aed27c5f]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:24:26 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:24:26 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.256s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] HCTL ('33', '-', '-', 0) found on session 23 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[89376970-af7f-4f51-9240-3f21c30379ee]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:26 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Searching for a device in session 23 and hctl ['33', '*', '*', 0] yield: None {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Searching for a device in session 23 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YCyASkTZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.286s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] <== connect_volume: return (1287ms) {'type': 'block', 'scsi_wwn': '23466366334383665', 'path': '/dev/sdb'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23466366334383665', 'path': '/dev/sdb'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:24:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1117599705',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1117599705',id=24,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-5xp7dr1d',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:24:16Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=3df97db3-fd6c-4be6-af32-7cde9dbcce75,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f6:fa:7c,bridge_name='br-int',has_traffic_filtering=True,id=6baf0637-1ee9-4ccb-93a3-264041fb73f3,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6baf0637-1e') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG nova.objects.instance [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'pci_devices' on Instance uuid 3df97db3-fd6c-4be6-af32-7cde9dbcce75 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:24:27 user nova-compute[71317]:   <uuid>3df97db3-fd6c-4be6-af32-7cde9dbcce75</uuid>
Apr 14 04:24:27 user nova-compute[71317]:   <name>instance-00000018</name>
Apr 14 04:24:27 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:24:27 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:24:27 user nova-compute[71317]:   <metadata>
Apr 14 04:24:27 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:24:27 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:24:27 user nova-compute[71317]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1117599705</nova:name>
Apr 14 04:24:27 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:24:26</nova:creationTime>
Apr 14 04:24:27 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:24:27 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:24:27 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:24:27 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:24:27 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:24:27 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:24:27 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:24:27 user nova-compute[71317]:       <nova:owner>
Apr 14 04:24:27 user nova-compute[71317]:         <nova:user uuid="6ec7103e06e643a6b9b2c908ba8d7b06">tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member</nova:user>
Apr 14 04:24:27 user nova-compute[71317]:         <nova:project uuid="d5995b934b0546c0983387e46e3b3df4">tempest-ServerBootFromVolumeStableRescueTest-247821842</nova:project>
Apr 14 04:24:27 user nova-compute[71317]:       </nova:owner>
Apr 14 04:24:27 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:24:27 user nova-compute[71317]:       <nova:ports>
Apr 14 04:24:27 user nova-compute[71317]:         <nova:port uuid="6baf0637-1ee9-4ccb-93a3-264041fb73f3">
Apr 14 04:24:27 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 14 04:24:27 user nova-compute[71317]:         </nova:port>
Apr 14 04:24:27 user nova-compute[71317]:       </nova:ports>
Apr 14 04:24:27 user nova-compute[71317]:     </nova:instance>
Apr 14 04:24:27 user nova-compute[71317]:   </metadata>
Apr 14 04:24:27 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:24:27 user nova-compute[71317]:     <system>
Apr 14 04:24:27 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:24:27 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:24:27 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:24:27 user nova-compute[71317]:       <entry name="serial">3df97db3-fd6c-4be6-af32-7cde9dbcce75</entry>
Apr 14 04:24:27 user nova-compute[71317]:       <entry name="uuid">3df97db3-fd6c-4be6-af32-7cde9dbcce75</entry>
Apr 14 04:24:27 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:24:27 user nova-compute[71317]:     </system>
Apr 14 04:24:27 user nova-compute[71317]:   </sysinfo>
Apr 14 04:24:27 user nova-compute[71317]:   <os>
Apr 14 04:24:27 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:24:27 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:24:27 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:24:27 user nova-compute[71317]:   </os>
Apr 14 04:24:27 user nova-compute[71317]:   <features>
Apr 14 04:24:27 user nova-compute[71317]:     <acpi/>
Apr 14 04:24:27 user nova-compute[71317]:     <apic/>
Apr 14 04:24:27 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:24:27 user nova-compute[71317]:   </features>
Apr 14 04:24:27 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:24:27 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:24:27 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:24:27 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:24:27 user nova-compute[71317]:   </clock>
Apr 14 04:24:27 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:24:27 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:24:27 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:24:27 user nova-compute[71317]:   </cpu>
Apr 14 04:24:27 user nova-compute[71317]:   <devices>
Apr 14 04:24:27 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:24:27 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:24:27 user nova-compute[71317]:       <source dev="/dev/sdb"/>
Apr 14 04:24:27 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:24:27 user nova-compute[71317]:       <serial>6e31b954-88d8-448b-bbb7-f19007a51866</serial>
Apr 14 04:24:27 user nova-compute[71317]:     </disk>
Apr 14 04:24:27 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:24:27 user nova-compute[71317]:       <mac address="fa:16:3e:f6:fa:7c"/>
Apr 14 04:24:27 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:24:27 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:24:27 user nova-compute[71317]:       <target dev="tap6baf0637-1e"/>
Apr 14 04:24:27 user nova-compute[71317]:     </interface>
Apr 14 04:24:27 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:24:27 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/console.log" append="off"/>
Apr 14 04:24:27 user nova-compute[71317]:     </serial>
Apr 14 04:24:27 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:24:27 user nova-compute[71317]:     <video>
Apr 14 04:24:27 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:24:27 user nova-compute[71317]:     </video>
Apr 14 04:24:27 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:24:27 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:24:27 user nova-compute[71317]:     </rng>
Apr 14 04:24:27 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:24:27 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:24:27 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:24:27 user nova-compute[71317]:     </memballoon>
Apr 14 04:24:27 user nova-compute[71317]:   </devices>
Apr 14 04:24:27 user nova-compute[71317]: </domain>
Apr 14 04:24:27 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:24:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1117599705',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1117599705',id=24,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-5xp7dr1d',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:24:16Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=3df97db3-fd6c-4be6-af32-7cde9dbcce75,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f6:fa:7c,bridge_name='br-int',has_traffic_filtering=True,id=6baf0637-1ee9-4ccb-93a3-264041fb73f3,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6baf0637-1e') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG os_vif [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f6:fa:7c,bridge_name='br-int',has_traffic_filtering=True,id=6baf0637-1ee9-4ccb-93a3-264041fb73f3,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6baf0637-1e') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap6baf0637-1e, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap6baf0637-1e, col_values=(('external_ids', {'iface-id': '6baf0637-1ee9-4ccb-93a3-264041fb73f3', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f6:fa:7c', 'vm-uuid': '3df97db3-fd6c-4be6-af32-7cde9dbcce75'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:27 user nova-compute[71317]: INFO os_vif [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f6:fa:7c,bridge_name='br-int',has_traffic_filtering=True,id=6baf0637-1ee9-4ccb-93a3-264041fb73f3,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6baf0637-1e')
Apr 14 04:24:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No VIF found with MAC fa:16:3e:f6:fa:7c, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:24:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "97392a47-53f0-4bde-977a-e2945332fe1f" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "97392a47-53f0-4bde-977a-e2945332fe1f" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG nova.objects.instance [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lazy-loading 'flavor' on Instance uuid 97392a47-53f0-4bde-977a-e2945332fe1f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:29 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Ignoring supplied device name: /dev/vdb
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "97392a47-53f0-4bde-977a-e2945332fe1f" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.055s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "97392a47-53f0-4bde-977a-e2945332fe1f" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "97392a47-53f0-4bde-977a-e2945332fe1f" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:29 user nova-compute[71317]: INFO nova.compute.manager [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Attaching volume de74dbb1-1657-4193-9520-50cfa823199b to /dev/vdb
Apr 14 04:24:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG nova.compute.manager [req-e0c16108-c8f6-4114-9b30-1b5ffe81327f req-47b092dd-c109-4922-9115-12ba53b6d936 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e0c16108-c8f6-4114-9b30-1b5ffe81327f req-47b092dd-c109-4922-9115-12ba53b6d936 service nova] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e0c16108-c8f6-4114-9b30-1b5ffe81327f req-47b092dd-c109-4922-9115-12ba53b6d936 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e0c16108-c8f6-4114-9b30-1b5ffe81327f req-47b092dd-c109-4922-9115-12ba53b6d936 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG nova.compute.manager [req-e0c16108-c8f6-4114-9b30-1b5ffe81327f req-47b092dd-c109-4922-9115-12ba53b6d936 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] No waiting events found dispatching network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:24:29 user nova-compute[71317]: WARNING nova.compute.manager [req-e0c16108-c8f6-4114-9b30-1b5ffe81327f req-47b092dd-c109-4922-9115-12ba53b6d936 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received unexpected event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 for instance with vm_state building and task_state spawning.
Apr 14 04:24:29 user nova-compute[71317]: DEBUG os_brick.utils [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[26a1186f-c0f8-4729-99f5-5702e09ad677]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4004b74-1e38-48b3-91ff-07f884904a13]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c0b7a6c-fefb-4f19-86ef-c5a0b54fd0ce]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ef54bb6-9cba-4483-a94e-c097292462d2]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:24:29 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:24:29 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[93670281-dbcb-41d3-ae77-4ce0de7be708]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG os_brick.utils [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] <== get_connector_properties: return (127ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:24:29 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Updating existing volume attachment record: eb2fea8e-c47a-458a-9fa5-f84c13b16e30 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:24:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:24:30 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:24:30 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Didn't find any instances for network info cache update. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 04:24:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:24:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:24:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:24:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:24:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:24:30 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:24:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446271.38929, 3df97db3-fd6c-4be6-af32-7cde9dbcce75 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:24:31 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] VM Resumed (Lifecycle Event)
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:24:31 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Instance spawned successfully.
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:24:31 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446271.395178, 3df97db3-fd6c-4be6-af32-7cde9dbcce75 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:24:31 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] VM Started (Lifecycle Event)
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:24:31 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:24:31 user nova-compute[71317]: INFO nova.compute.manager [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Took 5.05 seconds to spawn the instance on the hypervisor.
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.compute.manager [req-001142bc-ca35-4d09-a5af-d98df59f4307 req-a7b6d1e9-1a4d-4ad7-9843-9aba6df964c2 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-001142bc-ca35-4d09-a5af-d98df59f4307 req-a7b6d1e9-1a4d-4ad7-9843-9aba6df964c2 service nova] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-001142bc-ca35-4d09-a5af-d98df59f4307 req-a7b6d1e9-1a4d-4ad7-9843-9aba6df964c2 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-001142bc-ca35-4d09-a5af-d98df59f4307 req-a7b6d1e9-1a4d-4ad7-9843-9aba6df964c2 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:31 user nova-compute[71317]: DEBUG nova.compute.manager [req-001142bc-ca35-4d09-a5af-d98df59f4307 req-a7b6d1e9-1a4d-4ad7-9843-9aba6df964c2 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] No waiting events found dispatching network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:24:31 user nova-compute[71317]: WARNING nova.compute.manager [req-001142bc-ca35-4d09-a5af-d98df59f4307 req-a7b6d1e9-1a4d-4ad7-9843-9aba6df964c2 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received unexpected event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 for instance with vm_state building and task_state spawning.
Apr 14 04:24:31 user nova-compute[71317]: INFO nova.compute.manager [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Took 16.03 seconds to build instance.
Apr 14 04:24:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-cd573453-2dee-4d84-8124-4274af07943e tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.127s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:24:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:33 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YuLEGNAR', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:24:34 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:24:34 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:24:34 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260" returned: 21 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6c78a22-8a34-4706-8ebf-cbd570e8d849]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:24:34 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9fdd1097-360f-453c-9f29-5712f13c3871]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b] added
Apr 14 04:24:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0abefc5-d3a1-4f71-8cc7-2ea9663289cc]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8844b53f-1523-4255-8651-cf0b4c57ee84]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YuLEGNAR {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YuLEGNAR" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[77ed22cd-16aa-4055-9591-cda4b45bdfbe]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YuLEGNAR'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f19a8fa9-f9b6-4335-9cc0-00822b6c3386]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[732b338b-3cf1-46e6-b2a2-77abfb777b4f]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:24:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:24:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --login" returned: 0 in 0.070s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee2017ae-441d-4755-9eaa-a0ecde69e92e]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b, portal: 172.16.0.220,3260]
Apr 14 04:24:34 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b, portal: 172.16.0.220,3260] successful.
Apr 14 04:24:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e858cd95-ea68-464b-9864-01785f0b5b43]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.163s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f7c4605-92bd-456c-8f7c-ba98a7902ced]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:24:34 user nova-compute[71317]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b (non-flash)
Apr 14 04:24:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:24:34 user nova-compute[71317]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b (non-flash)
Apr 14 04:24:34 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.379s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] HCTL ('34', '-', '-', 0) found on session 24 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4645788-d770-4630-b5a7-d0d70f075abb]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Searching for a device in session 24 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YuLEGNAR', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.425s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] <== connect_volume: return (1429ms) {'type': 'block', 'scsi_wwn': '23738643433306262', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23738643433306262', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG nova.objects.instance [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lazy-loading 'flavor' on Instance uuid 97392a47-53f0-4bde-977a-e2945332fe1f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:24:35 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:24:35 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:24:35 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:24:35 user nova-compute[71317]:   <serial>de74dbb1-1657-4193-9520-50cfa823199b</serial>
Apr 14 04:24:35 user nova-compute[71317]: </disk>
Apr 14 04:24:35 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] No VIF found with MAC fa:16:3e:6e:4a:74, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-defcc02b-34a3-45f3-911a-b8757e910851 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "97392a47-53f0-4bde-977a-e2945332fe1f" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.837s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:38 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:24:38 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7187MB free_disk=26.326847076416016GB free_vcpus=-1 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 9b532a44-20b7-4590-babe-684c49598a62 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance da3b4651-aaa0-430d-85b8-c15de386d08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance d2e6cf32-080b-47b2-b4db-2b776ba85b38 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 97392a47-53f0-4bde-977a-e2945332fe1f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 46a5fd77-abec-4ab3-b519-7c649e63459c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance f593ceb0-de44-490b-b365-3a05a3acd066 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 5273d202-ffc5-43c6-bb4f-f6d1a6855271 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 13 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:24:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=2176MB phys_disk=40GB used_disk=12GB total_vcpus=12 used_vcpus=13 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:24:39 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:24:39 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:24:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:24:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.535s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Acquiring lock "46a5fd77-abec-4ab3-b519-7c649e63459c" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Acquiring lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:45 user nova-compute[71317]: INFO nova.compute.manager [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Terminating instance
Apr 14 04:24:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG nova.compute.manager [req-97fc71a0-784a-4d80-9401-d70986d991b9 req-ad27d33b-c0bb-4cfd-bc55-5de7735c963c service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Received event network-vif-unplugged-534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-97fc71a0-784a-4d80-9401-d70986d991b9 req-ad27d33b-c0bb-4cfd-bc55-5de7735c963c service nova] Acquiring lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-97fc71a0-784a-4d80-9401-d70986d991b9 req-ad27d33b-c0bb-4cfd-bc55-5de7735c963c service nova] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-97fc71a0-784a-4d80-9401-d70986d991b9 req-ad27d33b-c0bb-4cfd-bc55-5de7735c963c service nova] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG nova.compute.manager [req-97fc71a0-784a-4d80-9401-d70986d991b9 req-ad27d33b-c0bb-4cfd-bc55-5de7735c963c service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] No waiting events found dispatching network-vif-unplugged-534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG nova.compute.manager [req-97fc71a0-784a-4d80-9401-d70986d991b9 req-ad27d33b-c0bb-4cfd-bc55-5de7735c963c service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Received event network-vif-unplugged-534d2295-4787-4bd1-9e53-4cf6acbf4a2b for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:45 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Instance destroyed successfully.
Apr 14 04:24:45 user nova-compute[71317]: DEBUG nova.objects.instance [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lazy-loading 'resources' on Instance uuid 46a5fd77-abec-4ab3-b519-7c649e63459c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:22:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-1070759898',display_name='tempest-VolumesNegativeTest-instance-1070759898',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-1070759898',id=20,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:23:02Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3f45f01678f747268221c00f75924c69',ramdisk_id='',reservation_id='r-wncqohyy',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesNegativeTest-1984420239',owner_user_name='tempest-VolumesNegativeTest-1984420239-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:23:03Z,user_data=None,user_id='275086d3eec449678b6b5ba6f3f887b2',uuid=46a5fd77-abec-4ab3-b519-7c649e63459c,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "address": "fa:16:3e:9d:19:0d", "network": {"id": "b96b0659-2bac-40e7-b707-62f8a9fae7a7", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.87", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "d8b3c3fd68cb4219826c2a6fd8b21939", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap534d2295-47", "ovs_interfaceid": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Converting VIF {"id": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "address": "fa:16:3e:9d:19:0d", "network": {"id": "b96b0659-2bac-40e7-b707-62f8a9fae7a7", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.87", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "d8b3c3fd68cb4219826c2a6fd8b21939", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap534d2295-47", "ovs_interfaceid": "534d2295-4787-4bd1-9e53-4cf6acbf4a2b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9d:19:0d,bridge_name='br-int',has_traffic_filtering=True,id=534d2295-4787-4bd1-9e53-4cf6acbf4a2b,network=Network(b96b0659-2bac-40e7-b707-62f8a9fae7a7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap534d2295-47') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG os_vif [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9d:19:0d,bridge_name='br-int',has_traffic_filtering=True,id=534d2295-4787-4bd1-9e53-4cf6acbf4a2b,network=Network(b96b0659-2bac-40e7-b707-62f8a9fae7a7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap534d2295-47') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap534d2295-47, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:45 user nova-compute[71317]: INFO os_vif [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9d:19:0d,bridge_name='br-int',has_traffic_filtering=True,id=534d2295-4787-4bd1-9e53-4cf6acbf4a2b,network=Network(b96b0659-2bac-40e7-b707-62f8a9fae7a7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap534d2295-47')
Apr 14 04:24:45 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Deleting instance files /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c_del
Apr 14 04:24:45 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Deletion of /opt/stack/data/nova/instances/46a5fd77-abec-4ab3-b519-7c649e63459c_del complete
Apr 14 04:24:46 user nova-compute[71317]: INFO nova.compute.manager [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Took 0.66 seconds to destroy the instance on the hypervisor.
Apr 14 04:24:46 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:24:46 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:24:46 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:24:46 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:24:46 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Took 0.45 seconds to deallocate network for instance.
Apr 14 04:24:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:46 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:24:46 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:24:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.425s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:46 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Deleted allocations for instance 46a5fd77-abec-4ab3-b519-7c649e63459c
Apr 14 04:24:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-416794ea-9f01-4a9c-95f5-11f751b1b520 tempest-VolumesNegativeTest-1984420239 tempest-VolumesNegativeTest-1984420239-project-member] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.710s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-6e705004-b9e9-4107-afb1-86f913c52b2d req-d8790fc5-3d44-47a0-a7ce-126e464b74f2 service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Received event network-vif-plugged-534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6e705004-b9e9-4107-afb1-86f913c52b2d req-d8790fc5-3d44-47a0-a7ce-126e464b74f2 service nova] Acquiring lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6e705004-b9e9-4107-afb1-86f913c52b2d req-d8790fc5-3d44-47a0-a7ce-126e464b74f2 service nova] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6e705004-b9e9-4107-afb1-86f913c52b2d req-d8790fc5-3d44-47a0-a7ce-126e464b74f2 service nova] Lock "46a5fd77-abec-4ab3-b519-7c649e63459c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-6e705004-b9e9-4107-afb1-86f913c52b2d req-d8790fc5-3d44-47a0-a7ce-126e464b74f2 service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] No waiting events found dispatching network-vif-plugged-534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:24:47 user nova-compute[71317]: WARNING nova.compute.manager [req-6e705004-b9e9-4107-afb1-86f913c52b2d req-d8790fc5-3d44-47a0-a7ce-126e464b74f2 service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Received unexpected event network-vif-plugged-534d2295-4787-4bd1-9e53-4cf6acbf4a2b for instance with vm_state deleted and task_state None.
Apr 14 04:24:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-6e705004-b9e9-4107-afb1-86f913c52b2d req-d8790fc5-3d44-47a0-a7ce-126e464b74f2 service nova] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Received event network-vif-deleted-534d2295-4787-4bd1-9e53-4cf6acbf4a2b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "97392a47-53f0-4bde-977a-e2945332fe1f" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "97392a47-53f0-4bde-977a-e2945332fe1f" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:55 user nova-compute[71317]: INFO nova.compute.manager [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Detaching volume de74dbb1-1657-4193-9520-50cfa823199b
Apr 14 04:24:55 user nova-compute[71317]: INFO nova.virt.block_device [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Attempting to driver detach volume de74dbb1-1657-4193-9520-50cfa823199b from mountpoint /dev/vdb
Apr 14 04:24:55 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Attempting to detach device vdb from instance 97392a47-53f0-4bde-977a-e2945332fe1f from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:24:55 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:24:55 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:24:55 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:24:55 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:24:55 user nova-compute[71317]:   <serial>de74dbb1-1657-4193-9520-50cfa823199b</serial>
Apr 14 04:24:55 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:24:55 user nova-compute[71317]: </disk>
Apr 14 04:24:55 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:24:55 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Successfully detached device vdb from instance 97392a47-53f0-4bde-977a-e2945332fe1f from the persistent domain config.
Apr 14 04:24:55 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 97392a47-53f0-4bde-977a-e2945332fe1f from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:24:55 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:24:55 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:24:55 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:24:55 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:24:55 user nova-compute[71317]:   <serial>de74dbb1-1657-4193-9520-50cfa823199b</serial>
Apr 14 04:24:55 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:24:55 user nova-compute[71317]: </disk>
Apr 14 04:24:55 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:24:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "97392a47-53f0-4bde-977a-e2945332fe1f" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446296.3719304, 97392a47-53f0-4bde-977a-e2945332fe1f => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 97392a47-53f0-4bde-977a-e2945332fe1f {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:24:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Successfully detached device vdb from instance 97392a47-53f0-4bde-977a-e2945332fe1f from the live domain config.
Apr 14 04:24:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YuLEGNAR', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:24:56 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1bea169-9c8a-4e03-8cfd-85e7eb7a662e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:24:56 user nova-compute[71317]: No targets found.
Apr 14 04:24:56 user nova-compute[71317]: iSNS:
Apr 14 04:24:56 user nova-compute[71317]: No targets found.
Apr 14 04:24:56 user nova-compute[71317]: STATIC:
Apr 14 04:24:56 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:24:56 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:24:56 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:24:56 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b
Apr 14 04:24:56 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:24:56 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:24:56 user nova-compute[71317]: FIRMWARE:
Apr 14 04:24:56 user nova-compute[71317]: No targets found.
Apr 14 04:24:56 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:24:56 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:24:56 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c57633e0-9b1b-4536-a4c3-91fb36852d55]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b514013e-28d0-4952-8552-f33b214fb3b5]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:24:56 user nova-compute[71317]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b (non-flash)
Apr 14 04:24:56 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:24:56 user nova-compute[71317]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b (non-flash)
Apr 14 04:24:56 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84b41343a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2254e003-8fa7-4bc4-b8f1-b9dd1232fdf0]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[37ff8fca-cca3-4d9d-8a0a-41257bcdaebe]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f8ff7f4-11f7-47b0-8d64-a4aee6b787c9]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.045s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ccdab53-c605-44aa-a82a-c648d2da2ed4]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c5c8f8e5-77ff-4796-ad7f-3f60e560ab91]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --logout" returned: 0 in 0.053s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc2a1d37-2570-4510-8101-c0cf536ce77b]: (4, ('Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b, portal: 172.16.0.220,3260]\nLogout of [sid: 24, target: iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b, portal: 172.16.0.220,3260]
Apr 14 04:24:56 user nova-compute[71317]: Logout of [sid: 24, target: iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b, portal: 172.16.0.220,3260] successful.
Apr 14 04:24:56 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6d399c8-0a1e-47ab-9139-00a6b49b782d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.295s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] <== disconnect_volume: return (296ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:24:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:24:57 user nova-compute[71317]: DEBUG nova.objects.instance [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lazy-loading 'flavor' on Instance uuid 97392a47-53f0-4bde-977a-e2945332fe1f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6c88e981-9c2c-44b0-b2ea-f63b39c7488e tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "97392a47-53f0-4bde-977a-e2945332fe1f" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.723s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "97392a47-53f0-4bde-977a-e2945332fe1f" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 1.592s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:57 user nova-compute[71317]: INFO nova.compute.manager [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Terminating instance
Apr 14 04:24:57 user nova-compute[71317]: DEBUG nova.compute.manager [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:24:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-3befa2ba-34c6-43e3-bf8d-4a5d471580e2 req-d14e564d-2b0d-4046-b5b2-be2a80a0c167 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Received event network-vif-unplugged-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3befa2ba-34c6-43e3-bf8d-4a5d471580e2 req-d14e564d-2b0d-4046-b5b2-be2a80a0c167 service nova] Acquiring lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3befa2ba-34c6-43e3-bf8d-4a5d471580e2 req-d14e564d-2b0d-4046-b5b2-be2a80a0c167 service nova] Lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3befa2ba-34c6-43e3-bf8d-4a5d471580e2 req-d14e564d-2b0d-4046-b5b2-be2a80a0c167 service nova] Lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-3befa2ba-34c6-43e3-bf8d-4a5d471580e2 req-d14e564d-2b0d-4046-b5b2-be2a80a0c167 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] No waiting events found dispatching network-vif-unplugged-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-3befa2ba-34c6-43e3-bf8d-4a5d471580e2 req-d14e564d-2b0d-4046-b5b2-be2a80a0c167 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Received event network-vif-unplugged-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:24:58 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Instance destroyed successfully.
Apr 14 04:24:58 user nova-compute[71317]: DEBUG nova.objects.instance [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lazy-loading 'resources' on Instance uuid 97392a47-53f0-4bde-977a-e2945332fe1f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:22:38Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-929176901',display_name='tempest-VolumesSnapshotTestJSON-instance-929176901',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-929176901',id=19,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:22:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0848c9a67e2746e5b12b73920215b729',ramdisk_id='',reservation_id='r-p6ob5qid',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesSnapshotTestJSON-1226436274',owner_user_name='tempest-VolumesSnapshotTestJSON-1226436274-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:22:47Z,user_data=None,user_id='e4a70281d6734c15b3c4b1b00e4b5d44',uuid=97392a47-53f0-4bde-977a-e2945332fe1f,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "address": "fa:16:3e:6e:4a:74", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap32f6910c-73", "ovs_interfaceid": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Converting VIF {"id": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "address": "fa:16:3e:6e:4a:74", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap32f6910c-73", "ovs_interfaceid": "32f6910c-738f-4bbd-bd2a-a2d1b9bc261d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6e:4a:74,bridge_name='br-int',has_traffic_filtering=True,id=32f6910c-738f-4bbd-bd2a-a2d1b9bc261d,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap32f6910c-73') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_vif [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6e:4a:74,bridge_name='br-int',has_traffic_filtering=True,id=32f6910c-738f-4bbd-bd2a-a2d1b9bc261d,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap32f6910c-73') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap32f6910c-73, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:58 user nova-compute[71317]: INFO os_vif [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6e:4a:74,bridge_name='br-int',has_traffic_filtering=True,id=32f6910c-738f-4bbd-bd2a-a2d1b9bc261d,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap32f6910c-73')
Apr 14 04:24:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YuLEGNAR', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:24:58 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ca42e7d-b971-4846-b52b-6046063d0deb]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:24:58 user nova-compute[71317]: No targets found.
Apr 14 04:24:58 user nova-compute[71317]: iSNS:
Apr 14 04:24:58 user nova-compute[71317]: No targets found.
Apr 14 04:24:58 user nova-compute[71317]: STATIC:
Apr 14 04:24:58 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:24:58 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:24:58 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:24:58 user nova-compute[71317]: FIRMWARE:
Apr 14 04:24:58 user nova-compute[71317]: No targets found.
Apr 14 04:24:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:24:58 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:24:58 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:de74dbb1-1657-4193-9520-50cfa823199b', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[149a6826-6e6d-4ed8-a3bb-b126625ec9e8]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7ef6d10-a548-4ef3-ab98-00c6770df0c2]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:24:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:24:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84b4134e50>, {}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Disconnecting from: [] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.071s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] <== disconnect_volume: return (72ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:24:58 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Deleting instance files /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f_del
Apr 14 04:24:58 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Deletion of /opt/stack/data/nova/instances/97392a47-53f0-4bde-977a-e2945332fe1f_del complete
Apr 14 04:24:58 user nova-compute[71317]: INFO nova.compute.manager [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Took 0.95 seconds to destroy the instance on the hypervisor.
Apr 14 04:24:58 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:24:58 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:24:59 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:24:59 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Took 1.17 seconds to deallocate network for instance.
Apr 14 04:24:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-052aefe8-1b79-428c-bbcb-216845e8c44c req-9cbc5e5a-da87-4b35-acbe-25bf36357399 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Received event network-vif-deleted-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:24:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:24:59 user nova-compute[71317]: WARNING nova.volume.cinder [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Attachment eb2fea8e-c47a-458a-9fa5-f84c13b16e30 does not exist. Ignoring.: cinderclient.exceptions.NotFound: Volume attachment could not be found with filter: attachment_id = eb2fea8e-c47a-458a-9fa5-f84c13b16e30. (HTTP 404) (Request-ID: req-f839fc40-4c61-43f3-b84c-c9077706f806)
Apr 14 04:24:59 user nova-compute[71317]: INFO nova.compute.manager [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Took 0.03 seconds to detach 1 volumes for instance.
Apr 14 04:24:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:24:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:00 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:25:00 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:25:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.401s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-5bcb3ae4-9e93-46d3-a483-0ed406876121 req-32b64e7f-8e70-4922-8b0b-d856562d3e84 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Received event network-vif-plugged-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5bcb3ae4-9e93-46d3-a483-0ed406876121 req-32b64e7f-8e70-4922-8b0b-d856562d3e84 service nova] Acquiring lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5bcb3ae4-9e93-46d3-a483-0ed406876121 req-32b64e7f-8e70-4922-8b0b-d856562d3e84 service nova] Lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5bcb3ae4-9e93-46d3-a483-0ed406876121 req-32b64e7f-8e70-4922-8b0b-d856562d3e84 service nova] Lock "97392a47-53f0-4bde-977a-e2945332fe1f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-5bcb3ae4-9e93-46d3-a483-0ed406876121 req-32b64e7f-8e70-4922-8b0b-d856562d3e84 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] No waiting events found dispatching network-vif-plugged-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:25:00 user nova-compute[71317]: WARNING nova.compute.manager [req-5bcb3ae4-9e93-46d3-a483-0ed406876121 req-32b64e7f-8e70-4922-8b0b-d856562d3e84 service nova] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Received unexpected event network-vif-plugged-32f6910c-738f-4bbd-bd2a-a2d1b9bc261d for instance with vm_state deleted and task_state None.
Apr 14 04:25:00 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Deleted allocations for instance 97392a47-53f0-4bde-977a-e2945332fe1f
Apr 14 04:25:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-434df4c2-881f-4306-89d1-ca4b9064f991 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "97392a47-53f0-4bde-977a-e2945332fe1f" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.729s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:00 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446285.918692, 46a5fd77-abec-4ab3-b519-7c649e63459c => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:25:00 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] VM Stopped (Lifecycle Event)
Apr 14 04:25:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-0b4bb719-32da-401d-bf9e-5e5b92897220 None None] [instance: 46a5fd77-abec-4ab3-b519-7c649e63459c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:25:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "9b532a44-20b7-4590-babe-684c49598a62" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "9b532a44-20b7-4590-babe-684c49598a62" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "9b532a44-20b7-4590-babe-684c49598a62-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "9b532a44-20b7-4590-babe-684c49598a62-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "9b532a44-20b7-4590-babe-684c49598a62-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:10 user nova-compute[71317]: INFO nova.compute.manager [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Terminating instance
Apr 14 04:25:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-df6e188f-ea0f-478a-9b8b-0aba5bc82d40 req-1c09fd1b-a5f8-41c4-9aa5-2b03161b97cc service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Received event network-vif-unplugged-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-df6e188f-ea0f-478a-9b8b-0aba5bc82d40 req-1c09fd1b-a5f8-41c4-9aa5-2b03161b97cc service nova] Acquiring lock "9b532a44-20b7-4590-babe-684c49598a62-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-df6e188f-ea0f-478a-9b8b-0aba5bc82d40 req-1c09fd1b-a5f8-41c4-9aa5-2b03161b97cc service nova] Lock "9b532a44-20b7-4590-babe-684c49598a62-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-df6e188f-ea0f-478a-9b8b-0aba5bc82d40 req-1c09fd1b-a5f8-41c4-9aa5-2b03161b97cc service nova] Lock "9b532a44-20b7-4590-babe-684c49598a62-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-df6e188f-ea0f-478a-9b8b-0aba5bc82d40 req-1c09fd1b-a5f8-41c4-9aa5-2b03161b97cc service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] No waiting events found dispatching network-vif-unplugged-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-df6e188f-ea0f-478a-9b8b-0aba5bc82d40 req-1c09fd1b-a5f8-41c4-9aa5-2b03161b97cc service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Received event network-vif-unplugged-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:25:10 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Instance destroyed successfully.
Apr 14 04:25:10 user nova-compute[71317]: DEBUG nova.objects.instance [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lazy-loading 'resources' on Instance uuid 9b532a44-20b7-4590-babe-684c49598a62 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1687697238',display_name='tempest-ServersNegativeTestJSON-server-1687697238',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1687697238',id=2,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:16:39Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bae0fb4af555422a847620627dffeae8',ramdisk_id='',reservation_id='r-70d12ce1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServersNegativeTestJSON-406789078',owner_user_name='tempest-ServersNegativeTestJSON-406789078-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:16:40Z,user_data=None,user_id='ea68fb139b004f748cf1bcfb8d675a2e',uuid=9b532a44-20b7-4590-babe-684c49598a62,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "address": "fa:16:3e:47:9f:40", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc45f9c53-8d", "ovs_interfaceid": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Converting VIF {"id": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "address": "fa:16:3e:47:9f:40", "network": {"id": "a462dbfb-12e1-48c7-a797-87823269af2a", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-168891882-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bae0fb4af555422a847620627dffeae8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc45f9c53-8d", "ovs_interfaceid": "c45f9c53-8d5c-4b47-8e0a-ebb292cefb06", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:47:9f:40,bridge_name='br-int',has_traffic_filtering=True,id=c45f9c53-8d5c-4b47-8e0a-ebb292cefb06,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc45f9c53-8d') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG os_vif [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:47:9f:40,bridge_name='br-int',has_traffic_filtering=True,id=c45f9c53-8d5c-4b47-8e0a-ebb292cefb06,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc45f9c53-8d') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc45f9c53-8d, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:25:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:25:11 user nova-compute[71317]: INFO os_vif [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:47:9f:40,bridge_name='br-int',has_traffic_filtering=True,id=c45f9c53-8d5c-4b47-8e0a-ebb292cefb06,network=Network(a462dbfb-12e1-48c7-a797-87823269af2a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc45f9c53-8d')
Apr 14 04:25:11 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Deleting instance files /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62_del
Apr 14 04:25:11 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Deletion of /opt/stack/data/nova/instances/9b532a44-20b7-4590-babe-684c49598a62_del complete
Apr 14 04:25:11 user nova-compute[71317]: INFO nova.compute.manager [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Took 0.68 seconds to destroy the instance on the hypervisor.
Apr 14 04:25:11 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:25:11 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:25:11 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 9b532a44-20b7-4590-babe-684c49598a62] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:25:11 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:25:11 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Took 0.48 seconds to deallocate network for instance.
Apr 14 04:25:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-b91abdff-0c1c-4bde-a24d-bb4de7bfab30 req-d8bcae77-4e53-4cdc-b336-17568bfdb091 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Received event network-vif-deleted-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:11 user nova-compute[71317]: INFO nova.compute.manager [req-b91abdff-0c1c-4bde-a24d-bb4de7bfab30 req-d8bcae77-4e53-4cdc-b336-17568bfdb091 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Neutron deleted interface c45f9c53-8d5c-4b47-8e0a-ebb292cefb06; detaching it from the instance and deleting it from the info cache
Apr 14 04:25:11 user nova-compute[71317]: DEBUG nova.network.neutron [req-b91abdff-0c1c-4bde-a24d-bb4de7bfab30 req-d8bcae77-4e53-4cdc-b336-17568bfdb091 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:25:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-b91abdff-0c1c-4bde-a24d-bb4de7bfab30 req-d8bcae77-4e53-4cdc-b336-17568bfdb091 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Detach interface failed, port_id=c45f9c53-8d5c-4b47-8e0a-ebb292cefb06, reason: Instance 9b532a44-20b7-4590-babe-684c49598a62 could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:25:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:11 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:25:11 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:25:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.378s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:12 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Deleted allocations for instance 9b532a44-20b7-4590-babe-684c49598a62
Apr 14 04:25:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-368c135b-674b-4bad-9850-d373af76294b tempest-ServersNegativeTestJSON-406789078 tempest-ServersNegativeTestJSON-406789078-project-member] Lock "9b532a44-20b7-4590-babe-684c49598a62" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.724s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-c23d70ac-8c8d-4217-a4b7-11d85269c8bb req-05414f7c-1383-4a2f-8d95-6f79a5b833b0 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Received event network-vif-plugged-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c23d70ac-8c8d-4217-a4b7-11d85269c8bb req-05414f7c-1383-4a2f-8d95-6f79a5b833b0 service nova] Acquiring lock "9b532a44-20b7-4590-babe-684c49598a62-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c23d70ac-8c8d-4217-a4b7-11d85269c8bb req-05414f7c-1383-4a2f-8d95-6f79a5b833b0 service nova] Lock "9b532a44-20b7-4590-babe-684c49598a62-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c23d70ac-8c8d-4217-a4b7-11d85269c8bb req-05414f7c-1383-4a2f-8d95-6f79a5b833b0 service nova] Lock "9b532a44-20b7-4590-babe-684c49598a62-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-c23d70ac-8c8d-4217-a4b7-11d85269c8bb req-05414f7c-1383-4a2f-8d95-6f79a5b833b0 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] No waiting events found dispatching network-vif-plugged-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:25:12 user nova-compute[71317]: WARNING nova.compute.manager [req-c23d70ac-8c8d-4217-a4b7-11d85269c8bb req-05414f7c-1383-4a2f-8d95-6f79a5b833b0 service nova] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Received unexpected event network-vif-plugged-c45f9c53-8d5c-4b47-8e0a-ebb292cefb06 for instance with vm_state deleted and task_state None.
Apr 14 04:25:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:13 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446298.2784603, 97392a47-53f0-4bde-977a-e2945332fe1f => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:25:13 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] VM Stopped (Lifecycle Event)
Apr 14 04:25:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-438e9756-888b-461a-97a5-26b98016a273 None None] [instance: 97392a47-53f0-4bde-977a-e2945332fe1f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:25:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:25:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:25:15 user nova-compute[71317]: INFO nova.compute.claims [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Claim successful on node user
Apr 14 04:25:15 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:25:15 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:25:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.423s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:25:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:25:15 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:25:15 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:25:15 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG nova.policy [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '5234914aec8e443b8a38ac6b38729031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33320dc2f2c14ecfa5abf6574029c36c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:25:16 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Creating image(s)
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "/opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "/opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "/opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.152s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk 1073741824" returned: 0 in 0.054s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.192s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Checking if we can resize image /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Cannot resize image /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'migration_context' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Ensure instance console log exists: /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:16 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Successfully created port: e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:25:17 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Successfully updated port: e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:25:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:25:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquired lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:25:17 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:25:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-3b27c5f4-95ca-427e-94bd-9a88f58da432 req-bdd9fe1d-ed90-4b00-9221-5d37c19118fd service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-changed-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-3b27c5f4-95ca-427e-94bd-9a88f58da432 req-bdd9fe1d-ed90-4b00-9221-5d37c19118fd service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Refreshing instance network info cache due to event network-changed-e3868733-d538-4469-9a3e-c02014ec9066. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:25:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3b27c5f4-95ca-427e-94bd-9a88f58da432 req-bdd9fe1d-ed90-4b00-9221-5d37c19118fd service nova] Acquiring lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:25:17 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updating instance_info_cache with network_info: [{"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Releasing lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Instance network_info: |[{"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3b27c5f4-95ca-427e-94bd-9a88f58da432 req-bdd9fe1d-ed90-4b00-9221-5d37c19118fd service nova] Acquired lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.network.neutron [req-3b27c5f4-95ca-427e-94bd-9a88f58da432 req-bdd9fe1d-ed90-4b00-9221-5d37c19118fd service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Refreshing network info cache for port e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Start _get_guest_xml network_info=[{"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:25:18 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:25:18 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:25:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1081038034',display_name='tempest-ServerRescueNegativeTestJSON-server-1081038034',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1081038034',id=25,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCb+23dydPc8XvPesmHXUsmKMTGTpSa5GCZHGQJYeeij6pIqTWqaFg9yzcA/VhrLxaLND3aQCfGfHGfvo53ucWJsjWgHFiCEptESTMFy12g7udsn7vnD6U5hRBURzXxHFg==',key_name='tempest-keypair-1320453961',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='33320dc2f2c14ecfa5abf6574029c36c',ramdisk_id='',reservation_id='r-ym9z46z3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-508041341',owner_user_name='tempest-ServerRescueNegativeTestJSON-508041341-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:25:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5234914aec8e443b8a38ac6b38729031',uuid=cab84e5c-99ba-4e61-bf68-0a02b31ded5c,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converting VIF {"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:71:08:35,bridge_name='br-int',has_traffic_filtering=True,id=e3868733-d538-4469-9a3e-c02014ec9066,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3868733-d5') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'pci_devices' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:25:18 user nova-compute[71317]:   <uuid>cab84e5c-99ba-4e61-bf68-0a02b31ded5c</uuid>
Apr 14 04:25:18 user nova-compute[71317]:   <name>instance-00000019</name>
Apr 14 04:25:18 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:25:18 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:25:18 user nova-compute[71317]:   <metadata>
Apr 14 04:25:18 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:25:18 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:25:18 user nova-compute[71317]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-1081038034</nova:name>
Apr 14 04:25:18 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:25:18</nova:creationTime>
Apr 14 04:25:18 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:25:18 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:25:18 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:25:18 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:25:18 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:25:18 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:25:18 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:25:18 user nova-compute[71317]:       <nova:owner>
Apr 14 04:25:18 user nova-compute[71317]:         <nova:user uuid="5234914aec8e443b8a38ac6b38729031">tempest-ServerRescueNegativeTestJSON-508041341-project-member</nova:user>
Apr 14 04:25:18 user nova-compute[71317]:         <nova:project uuid="33320dc2f2c14ecfa5abf6574029c36c">tempest-ServerRescueNegativeTestJSON-508041341</nova:project>
Apr 14 04:25:18 user nova-compute[71317]:       </nova:owner>
Apr 14 04:25:18 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:25:18 user nova-compute[71317]:       <nova:ports>
Apr 14 04:25:18 user nova-compute[71317]:         <nova:port uuid="e3868733-d538-4469-9a3e-c02014ec9066">
Apr 14 04:25:18 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 04:25:18 user nova-compute[71317]:         </nova:port>
Apr 14 04:25:18 user nova-compute[71317]:       </nova:ports>
Apr 14 04:25:18 user nova-compute[71317]:     </nova:instance>
Apr 14 04:25:18 user nova-compute[71317]:   </metadata>
Apr 14 04:25:18 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:25:18 user nova-compute[71317]:     <system>
Apr 14 04:25:18 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:25:18 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:25:18 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:25:18 user nova-compute[71317]:       <entry name="serial">cab84e5c-99ba-4e61-bf68-0a02b31ded5c</entry>
Apr 14 04:25:18 user nova-compute[71317]:       <entry name="uuid">cab84e5c-99ba-4e61-bf68-0a02b31ded5c</entry>
Apr 14 04:25:18 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:25:18 user nova-compute[71317]:     </system>
Apr 14 04:25:18 user nova-compute[71317]:   </sysinfo>
Apr 14 04:25:18 user nova-compute[71317]:   <os>
Apr 14 04:25:18 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:25:18 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:25:18 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:25:18 user nova-compute[71317]:   </os>
Apr 14 04:25:18 user nova-compute[71317]:   <features>
Apr 14 04:25:18 user nova-compute[71317]:     <acpi/>
Apr 14 04:25:18 user nova-compute[71317]:     <apic/>
Apr 14 04:25:18 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:25:18 user nova-compute[71317]:   </features>
Apr 14 04:25:18 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:25:18 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:25:18 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:25:18 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:25:18 user nova-compute[71317]:   </clock>
Apr 14 04:25:18 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:25:18 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:25:18 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:25:18 user nova-compute[71317]:   </cpu>
Apr 14 04:25:18 user nova-compute[71317]:   <devices>
Apr 14 04:25:18 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:25:18 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:25:18 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk"/>
Apr 14 04:25:18 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:25:18 user nova-compute[71317]:     </disk>
Apr 14 04:25:18 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:25:18 user nova-compute[71317]:       <mac address="fa:16:3e:71:08:35"/>
Apr 14 04:25:18 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:25:18 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:25:18 user nova-compute[71317]:       <target dev="tape3868733-d5"/>
Apr 14 04:25:18 user nova-compute[71317]:     </interface>
Apr 14 04:25:18 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:25:18 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/console.log" append="off"/>
Apr 14 04:25:18 user nova-compute[71317]:     </serial>
Apr 14 04:25:18 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:25:18 user nova-compute[71317]:     <video>
Apr 14 04:25:18 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:25:18 user nova-compute[71317]:     </video>
Apr 14 04:25:18 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:25:18 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:25:18 user nova-compute[71317]:     </rng>
Apr 14 04:25:18 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:25:18 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:25:18 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:25:18 user nova-compute[71317]:     </memballoon>
Apr 14 04:25:18 user nova-compute[71317]:   </devices>
Apr 14 04:25:18 user nova-compute[71317]: </domain>
Apr 14 04:25:18 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:25:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1081038034',display_name='tempest-ServerRescueNegativeTestJSON-server-1081038034',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1081038034',id=25,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCb+23dydPc8XvPesmHXUsmKMTGTpSa5GCZHGQJYeeij6pIqTWqaFg9yzcA/VhrLxaLND3aQCfGfHGfvo53ucWJsjWgHFiCEptESTMFy12g7udsn7vnD6U5hRBURzXxHFg==',key_name='tempest-keypair-1320453961',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='33320dc2f2c14ecfa5abf6574029c36c',ramdisk_id='',reservation_id='r-ym9z46z3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-508041341',owner_user_name='tempest-ServerRescueNegativeTestJSON-508041341-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:25:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5234914aec8e443b8a38ac6b38729031',uuid=cab84e5c-99ba-4e61-bf68-0a02b31ded5c,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converting VIF {"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:71:08:35,bridge_name='br-int',has_traffic_filtering=True,id=e3868733-d538-4469-9a3e-c02014ec9066,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3868733-d5') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG os_vif [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:71:08:35,bridge_name='br-int',has_traffic_filtering=True,id=e3868733-d538-4469-9a3e-c02014ec9066,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3868733-d5') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape3868733-d5, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape3868733-d5, col_values=(('external_ids', {'iface-id': 'e3868733-d538-4469-9a3e-c02014ec9066', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:71:08:35', 'vm-uuid': 'cab84e5c-99ba-4e61-bf68-0a02b31ded5c'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:18 user nova-compute[71317]: INFO os_vif [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:71:08:35,bridge_name='br-int',has_traffic_filtering=True,id=e3868733-d538-4469-9a3e-c02014ec9066,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3868733-d5')
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No VIF found with MAC fa:16:3e:71:08:35, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.network.neutron [req-3b27c5f4-95ca-427e-94bd-9a88f58da432 req-bdd9fe1d-ed90-4b00-9221-5d37c19118fd service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updated VIF entry in instance network info cache for port e3868733-d538-4469-9a3e-c02014ec9066. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG nova.network.neutron [req-3b27c5f4-95ca-427e-94bd-9a88f58da432 req-bdd9fe1d-ed90-4b00-9221-5d37c19118fd service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updating instance_info_cache with network_info: [{"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:25:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3b27c5f4-95ca-427e-94bd-9a88f58da432 req-bdd9fe1d-ed90-4b00-9221-5d37c19118fd service nova] Releasing lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:25:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-3b27d63d-794f-46f7-bb84-426f778b8a9a req-6e230a6b-0430-4659-9166-20d61eb26bcb service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3b27d63d-794f-46f7-bb84-426f778b8a9a req-6e230a6b-0430-4659-9166-20d61eb26bcb service nova] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3b27d63d-794f-46f7-bb84-426f778b8a9a req-6e230a6b-0430-4659-9166-20d61eb26bcb service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3b27d63d-794f-46f7-bb84-426f778b8a9a req-6e230a6b-0430-4659-9166-20d61eb26bcb service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-3b27d63d-794f-46f7-bb84-426f778b8a9a req-6e230a6b-0430-4659-9166-20d61eb26bcb service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] No waiting events found dispatching network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:25:19 user nova-compute[71317]: WARNING nova.compute.manager [req-3b27d63d-794f-46f7-bb84-426f778b8a9a req-6e230a6b-0430-4659-9166-20d61eb26bcb service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received unexpected event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 for instance with vm_state building and task_state spawning.
Apr 14 04:25:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446321.712752, cab84e5c-99ba-4e61-bf68-0a02b31ded5c => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:25:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] VM Resumed (Lifecycle Event)
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:25:21 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Instance spawned successfully.
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:25:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446321.7144482, cab84e5c-99ba-4e61-bf68-0a02b31ded5c => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:25:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] VM Started (Lifecycle Event)
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:25:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:25:21 user nova-compute[71317]: INFO nova.compute.manager [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Took 5.75 seconds to spawn the instance on the hypervisor.
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-39861b9d-85d9-4118-a34d-53e1dcb57ad8 req-ed0dd638-a582-4bde-92c9-dd6897cd5f12 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-39861b9d-85d9-4118-a34d-53e1dcb57ad8 req-ed0dd638-a582-4bde-92c9-dd6897cd5f12 service nova] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-39861b9d-85d9-4118-a34d-53e1dcb57ad8 req-ed0dd638-a582-4bde-92c9-dd6897cd5f12 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-39861b9d-85d9-4118-a34d-53e1dcb57ad8 req-ed0dd638-a582-4bde-92c9-dd6897cd5f12 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-39861b9d-85d9-4118-a34d-53e1dcb57ad8 req-ed0dd638-a582-4bde-92c9-dd6897cd5f12 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] No waiting events found dispatching network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:25:21 user nova-compute[71317]: WARNING nova.compute.manager [req-39861b9d-85d9-4118-a34d-53e1dcb57ad8 req-ed0dd638-a582-4bde-92c9-dd6897cd5f12 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received unexpected event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 for instance with vm_state building and task_state spawning.
Apr 14 04:25:21 user nova-compute[71317]: INFO nova.compute.manager [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Took 6.50 seconds to build instance.
Apr 14 04:25:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2c104f98-c90d-4be3-bfbf-947f88452ec3 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.623s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:25 user nova-compute[71317]: INFO nova.compute.manager [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Terminating instance
Apr 14 04:25:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:25:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:25 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446310.966106, 9b532a44-20b7-4590-babe-684c49598a62 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:25:25 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 9b532a44-20b7-4590-babe-684c49598a62] VM Stopped (Lifecycle Event)
Apr 14 04:25:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a5a87ce7-3e8d-4452-9227-9547f46265c4 None None] [instance: 9b532a44-20b7-4590-babe-684c49598a62] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-c3b3e565-9d58-4d08-bcbe-3e4c9eac120a req-372abaa4-6ddd-4c99-8356-83e4f52d54b3 service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Received event network-vif-unplugged-d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c3b3e565-9d58-4d08-bcbe-3e4c9eac120a req-372abaa4-6ddd-4c99-8356-83e4f52d54b3 service nova] Acquiring lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c3b3e565-9d58-4d08-bcbe-3e4c9eac120a req-372abaa4-6ddd-4c99-8356-83e4f52d54b3 service nova] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c3b3e565-9d58-4d08-bcbe-3e4c9eac120a req-372abaa4-6ddd-4c99-8356-83e4f52d54b3 service nova] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-c3b3e565-9d58-4d08-bcbe-3e4c9eac120a req-372abaa4-6ddd-4c99-8356-83e4f52d54b3 service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] No waiting events found dispatching network-vif-unplugged-d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-c3b3e565-9d58-4d08-bcbe-3e4c9eac120a req-372abaa4-6ddd-4c99-8356-83e4f52d54b3 service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Received event network-vif-unplugged-d2a30aa1-7a43-4150-97c4-f312dea34ae7 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG nova.objects.instance [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'flavor' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.062s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:26 user nova-compute[71317]: INFO nova.compute.manager [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Attaching volume 9cdf2c58-c295-483e-b766-13e01129ede2 to /dev/vdb
Apr 14 04:25:26 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Instance destroyed successfully.
Apr 14 04:25:26 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lazy-loading 'resources' on Instance uuid 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:23:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1021322658',display_name='tempest-VolumesAdminNegativeTest-server-1021322658',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1021322658',id=21,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:23:39Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3d389bb78d2e40b9b07d2d3ecdb7566b',ramdisk_id='',reservation_id='r-d1yz9bco',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesAdminNegativeTest-618964915',owner_user_name='tempest-VolumesAdminNegativeTest-618964915-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:23:40Z,user_data=None,user_id='d2561bbc0c6d447da333ffdbb9fdc5f4',uuid=7ffb8a8e-b24f-46e8-84c4-130e9f18cf55,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "address": "fa:16:3e:4b:ba:b4", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd2a30aa1-7a", "ovs_interfaceid": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Converting VIF {"id": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "address": "fa:16:3e:4b:ba:b4", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd2a30aa1-7a", "ovs_interfaceid": "d2a30aa1-7a43-4150-97c4-f312dea34ae7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:4b:ba:b4,bridge_name='br-int',has_traffic_filtering=True,id=d2a30aa1-7a43-4150-97c4-f312dea34ae7,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd2a30aa1-7a') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG os_vif [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:4b:ba:b4,bridge_name='br-int',has_traffic_filtering=True,id=d2a30aa1-7a43-4150-97c4-f312dea34ae7,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd2a30aa1-7a') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapd2a30aa1-7a, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:25:26 user nova-compute[71317]: INFO os_vif [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:4b:ba:b4,bridge_name='br-int',has_traffic_filtering=True,id=d2a30aa1-7a43-4150-97c4-f312dea34ae7,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd2a30aa1-7a')
Apr 14 04:25:26 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Deleting instance files /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55_del
Apr 14 04:25:26 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Deletion of /opt/stack/data/nova/instances/7ffb8a8e-b24f-46e8-84c4-130e9f18cf55_del complete
Apr 14 04:25:26 user nova-compute[71317]: DEBUG os_brick.utils [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d642614f-47e2-4a2c-9143-434833c6186b]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4207b257-5043-4b4d-85cd-2fa15b6f160b]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a64557f-f3e0-464e-91f6-8e39ceb1f763]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[689aa3a7-62d3-4b60-9a7b-ebee99964870]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:26 user nova-compute[71317]: INFO nova.compute.manager [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Took 0.97 seconds to destroy the instance on the hypervisor.
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:25:26 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:25:26 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[42f6821b-c19e-4408-a341-36e2ebeb5440]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG os_brick.utils [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== get_connector_properties: return (145ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:25:26 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating existing volume attachment record: 01070649-9cf2-47ee-8a10-1fc0171f4573 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:25:27 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:25:27 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Took 0.70 seconds to deallocate network for instance.
Apr 14 04:25:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-8efab41a-4d79-407f-958f-f5a23bc85fe5 req-55d10506-6338-40af-b3df-9749edd562b6 service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Received event network-vif-deleted-d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:27 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:25:27 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:25:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.364s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:27 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Deleted allocations for instance 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55
Apr 14 04:25:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5eb2850a-dcbe-4623-9f9e-c7cc838657d1 tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.250s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-bdc63560-eaef-4b3e-920a-4afeb81b8a73 req-bd3ab1ad-8aa2-4388-b397-a2a63d69bdc8 service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Received event network-vif-plugged-d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bdc63560-eaef-4b3e-920a-4afeb81b8a73 req-bd3ab1ad-8aa2-4388-b397-a2a63d69bdc8 service nova] Acquiring lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bdc63560-eaef-4b3e-920a-4afeb81b8a73 req-bd3ab1ad-8aa2-4388-b397-a2a63d69bdc8 service nova] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bdc63560-eaef-4b3e-920a-4afeb81b8a73 req-bd3ab1ad-8aa2-4388-b397-a2a63d69bdc8 service nova] Lock "7ffb8a8e-b24f-46e8-84c4-130e9f18cf55-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-bdc63560-eaef-4b3e-920a-4afeb81b8a73 req-bd3ab1ad-8aa2-4388-b397-a2a63d69bdc8 service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] No waiting events found dispatching network-vif-plugged-d2a30aa1-7a43-4150-97c4-f312dea34ae7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:25:28 user nova-compute[71317]: WARNING nova.compute.manager [req-bdc63560-eaef-4b3e-920a-4afeb81b8a73 req-bd3ab1ad-8aa2-4388-b397-a2a63d69bdc8 service nova] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Received unexpected event network-vif-plugged-d2a30aa1-7a43-4150-97c4-f312dea34ae7 for instance with vm_state deleted and task_state None.
Apr 14 04:25:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:25:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:25:30 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:25:30 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:25:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:25:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:25:30 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:25:30 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lazy-loading 'info_cache' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updating instance_info_cache with network_info: [{"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-da3b4651-aaa0-430d-85b8-c15de386d08f" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MjAZDWGc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:25:31 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:25:31 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:25:31 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7afda3b-a9f0-42f4-90ea-e4315b28974a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:25:31 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d22f242-fa5b-4a31-9524-33612a91c445]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2] added
Apr 14 04:25:31 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9322b2ad-d417-4bce-b527-0228a7b7f26d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[144f492b-e690-46ea-aa15-187a28a0f0fc]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v MjAZDWGc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v MjAZDWGc" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dde148da-7b79-403f-9480-19ed7e5452ad]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'MjAZDWGc'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b3c8e229-df2f-4fa2-b545-93c2f2f6cf66]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1ec44ec-1ceb-403e-a2d9-e5c20cdcede2]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:31 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:31 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --login" returned: 0 in 0.068s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[de7642cb-1b91-400a-83be-5f84f32bf17f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260]
Apr 14 04:25:31 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260] successful.
Apr 14 04:25:31 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[78ef5a62-af22-49dc-8ffd-7df94ae0785a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.036s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc6ed73a-caaa-4343-88a3-94b076d05665]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:31 user nova-compute[71317]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:25:31 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:31 user nova-compute[71317]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:25:31 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.295s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] HCTL ('34', '-', '-', 0) found on session 25 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fd942bd-c73c-402c-9f66-a322d782ab6f]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Searching for a device in session 25 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:25:31 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MjAZDWGc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:25:32 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.328s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:25:32 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== connect_volume: return (1330ms) {'type': 'block', 'scsi_wwn': '23661646666666433', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:25:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23661646666666433', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:25:32 user nova-compute[71317]: DEBUG nova.objects.instance [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'flavor' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:25:32 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:25:32 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:25:32 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:25:32 user nova-compute[71317]:   <serial>9cdf2c58-c295-483e-b766-13e01129ede2</serial>
Apr 14 04:25:32 user nova-compute[71317]: </disk>
Apr 14 04:25:32 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:25:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:33 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:25:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:25:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:25:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No VIF found with MAC fa:16:3e:5d:8f:83, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:25:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e14ca792-7652-4305-8e06-f9fe36de7f49 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.116s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:34 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:25:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:34 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:25:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.177s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.171s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Stashing vm_state: active {{(pid=71317) _prep_resize /opt/stack/nova/nova/compute/manager.py:5499}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: waited 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG nova.objects.instance [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'pci_requests' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:25:36 user nova-compute[71317]: INFO nova.compute.claims [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Claim successful on node user
Apr 14 04:25:36 user nova-compute[71317]: DEBUG nova.objects.instance [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'resources' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG nova.objects.instance [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'pci_devices' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:36 user nova-compute[71317]: INFO nova.compute.resource_tracker [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating resource usage from migration c4903c05-a131-4f02-b6b6-805ce86d54cd
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: held 0.656s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:36 user nova-compute[71317]: INFO nova.compute.manager [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Migrating
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquired lock "compute-rpcapi-router" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:25:36 user nova-compute[71317]: INFO nova.compute.rpcapi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Automatically selected compute RPC version 6.2 from minimum service version 66
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Releasing lock "compute-rpcapi-router" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquired lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG nova.network.neutron [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:25:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:37 user nova-compute[71317]: DEBUG nova.network.neutron [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating instance_info_cache with network_info: [{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:25:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Releasing lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:25:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Starting migrate_disk_and_power_off {{(pid=71317) migrate_disk_and_power_off /opt/stack/nova/nova/virt/libvirt/driver.py:11460}}
Apr 14 04:25:38 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:25:38 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7559MB free_disk=26.373153686523438GB free_vcpus=2 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Applying migration context for instance a0a20aeb-ed06-4c65-b87d-74f8f825457d as it has an incoming, in-progress migration c4903c05-a131-4f02-b6b6-805ce86d54cd. Migration status is migrating {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:950}}
Apr 14 04:25:38 user nova-compute[71317]: INFO nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating resource usage from migration c4903c05-a131-4f02-b6b6-805ce86d54cd
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance da3b4651-aaa0-430d-85b8-c15de386d08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance d2e6cf32-080b-47b2-b4db-2b776ba85b38 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance f593ceb0-de44-490b-b365-3a05a3acd066 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 5273d202-ffc5-43c6-bb4f-f6d1a6855271 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Migration c4903c05-a131-4f02-b6b6-805ce86d54cd is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1637}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1984MB phys_disk=40GB used_disk=10GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-93f59faf-7494-447c-81ae-92cbf60a4161 req-bfdcae2e-97c3-45f3-a101-909b0b5d57c0 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-unplugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-93f59faf-7494-447c-81ae-92cbf60a4161 req-bfdcae2e-97c3-45f3-a101-909b0b5d57c0 service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-93f59faf-7494-447c-81ae-92cbf60a4161 req-bfdcae2e-97c3-45f3-a101-909b0b5d57c0 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-93f59faf-7494-447c-81ae-92cbf60a4161 req-bfdcae2e-97c3-45f3-a101-909b0b5d57c0 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-93f59faf-7494-447c-81ae-92cbf60a4161 req-bfdcae2e-97c3-45f3-a101-909b0b5d57c0 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-unplugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:25:38 user nova-compute[71317]: WARNING nova.compute.manager [req-93f59faf-7494-447c-81ae-92cbf60a4161 req-bfdcae2e-97c3-45f3-a101-909b0b5d57c0 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-unplugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state active and task_state resize_migrating.
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.553s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:38 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance destroyed successfully.
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1350614351',display_name='tempest-ServerActionsTestJSON-server-1398117926',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1350614351',id=6,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:20:59Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-789bnjb3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='resize_migrating',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:25:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=a0a20aeb-ed06-4c65-b87d-74f8f825457d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:5d:8f:83"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:5d:8f:83"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG os_vif [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3a3e3b13-7b, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:38 user nova-compute[71317]: INFO os_vif [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b')
Apr 14 04:25:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MjAZDWGc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:25:38 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:25:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:25:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.037s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[07d25a39-f47f-4cb5-bf55-0aa0e0f0d742]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:25:39 user nova-compute[71317]: No targets found.
Apr 14 04:25:39 user nova-compute[71317]: iSNS:
Apr 14 04:25:39 user nova-compute[71317]: No targets found.
Apr 14 04:25:39 user nova-compute[71317]: STATIC:
Apr 14 04:25:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:25:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:25:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:25:39 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:25:39 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:25:39 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:25:39 user nova-compute[71317]: FIRMWARE:
Apr 14 04:25:39 user nova-compute[71317]: No targets found.
Apr 14 04:25:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:25:39 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:25:39 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2bff0c2d-4dcb-4cfb-803b-a1ac27d3e2d1]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8623b3e5-90ab-43d4-a214-c91c528f36a2]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:39 user nova-compute[71317]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:25:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:39 user nova-compute[71317]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:25:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d18ab910>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b664f177-ba1a-44ef-97ff-11579e375d31]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c8aaab6-baa4-4987-b4fd-8041dd21f93a]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c9ae9b4-bcf5-432d-a896-a512e50b9185]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.054s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bab77456-c987-49a6-9afc-360bf53256c0]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d660d16b-85e8-4c32-b40e-1f57556e62fa]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-c7d74ffb-ad5e-485c-a485-6a6c16e06ce5 req-9a938db4-ef5c-4836-a40e-9795a252f42b service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Received event network-changed-dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-c7d74ffb-ad5e-485c-a485-6a6c16e06ce5 req-9a938db4-ef5c-4836-a40e-9795a252f42b service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Refreshing instance network info cache due to event network-changed-dd12687e-ca27-4fba-9653-4fa1af3608a8. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c7d74ffb-ad5e-485c-a485-6a6c16e06ce5 req-9a938db4-ef5c-4836-a40e-9795a252f42b service nova] Acquiring lock "refresh_cache-f593ceb0-de44-490b-b365-3a05a3acd066" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c7d74ffb-ad5e-485c-a485-6a6c16e06ce5 req-9a938db4-ef5c-4836-a40e-9795a252f42b service nova] Acquired lock "refresh_cache-f593ceb0-de44-490b-b365-3a05a3acd066" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG nova.network.neutron [req-c7d74ffb-ad5e-485c-a485-6a6c16e06ce5 req-9a938db4-ef5c-4836-a40e-9795a252f42b service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Refreshing network info cache for port dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --logout" returned: 0 in 0.080s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8262b665-3883-42b8-81a1-9f3d9d867826]: (4, ('Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260]\nLogout of [sid: 25, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260]
Apr 14 04:25:39 user nova-compute[71317]: Logout of [sid: 25, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260] successful.
Apr 14 04:25:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6844699f-b399-45b5-8ef9-44256c25d132]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.658s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== disconnect_volume: return (660ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG nova.objects.instance [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'flavor' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.101s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): cp -r /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d_resize/disk /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:39 user nova-compute[71317]: INFO nova.compute.manager [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attaching volume 8ee7c552-e40e-4341-992f-722c82128913 to /dev/vdb
Apr 14 04:25:40 user nova-compute[71317]: DEBUG os_brick.utils [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f809bbd7-fe06-46c9-972c-5c93a554e69d]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd6b87d1-5b49-4422-8eca-1afae89cfab1]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5e8dfe2-7975-440c-9007-e5f394e50698]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2eb1fb04-dc3c-40ae-b590-a6fd7bb4f67e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:25:40 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[554532bd-07b2-46cc-8e36-78b33e81a8c4]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG os_brick.utils [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] <== get_connector_properties: return (122ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Updating existing volume attachment record: cd50f445-165e-4fa4-94e8-183e38b93c10 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "cp -r /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d_resize/disk /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk" returned: 0 in 0.327s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): cp -r /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d_resize/disk.info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk.info {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "cp -r /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d_resize/disk.info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk.info" returned: 0 in 0.040s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-15a6fd4d-ca60-4e17-bd72-84df08646fdf req-333385cf-b594-42a2-9dce-af8dd624cd72 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-15a6fd4d-ca60-4e17-bd72-84df08646fdf req-333385cf-b594-42a2-9dce-af8dd624cd72 service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-15a6fd4d-ca60-4e17-bd72-84df08646fdf req-333385cf-b594-42a2-9dce-af8dd624cd72 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-15a6fd4d-ca60-4e17-bd72-84df08646fdf req-333385cf-b594-42a2-9dce-af8dd624cd72 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-15a6fd4d-ca60-4e17-bd72-84df08646fdf req-333385cf-b594-42a2-9dce-af8dd624cd72 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:25:40 user nova-compute[71317]: WARNING nova.compute.manager [req-15a6fd4d-ca60-4e17-bd72-84df08646fdf req-333385cf-b594-42a2-9dce-af8dd624cd72 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state active and task_state resize_migrating.
Apr 14 04:25:40 user nova-compute[71317]: DEBUG nova.network.neutron [req-c7d74ffb-ad5e-485c-a485-6a6c16e06ce5 req-9a938db4-ef5c-4836-a40e-9795a252f42b service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Updated VIF entry in instance network info cache for port dd12687e-ca27-4fba-9653-4fa1af3608a8. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG nova.network.neutron [req-c7d74ffb-ad5e-485c-a485-6a6c16e06ce5 req-9a938db4-ef5c-4836-a40e-9795a252f42b service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Updating instance_info_cache with network_info: [{"id": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "address": "fa:16:3e:17:59:61", "network": {"id": "694b7c0a-cb94-4b83-99aa-03c94c049086", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-383837474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.242", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "39731dbb51484eff890a74b07241ba75", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd12687e-ca", "ovs_interfaceid": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:25:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c7d74ffb-ad5e-485c-a485-6a6c16e06ce5 req-9a938db4-ef5c-4836-a40e-9795a252f42b service nova] Releasing lock "refresh_cache-f593ceb0-de44-490b-b365-3a05a3acd066" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:25:41 user nova-compute[71317]: DEBUG nova.network.neutron [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Port 3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 binding to destination host user is already ACTIVE {{(pid=71317) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3161}}
Apr 14 04:25:41 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446326.5515723, 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:25:41 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] VM Stopped (Lifecycle Event)
Apr 14 04:25:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-31040fb6-9750-4941-a726-6a9f488637e7 None None] [instance: 7ffb8a8e-b24f-46e8-84c4-130e9f18cf55] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:25:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:25:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquired lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:25:41 user nova-compute[71317]: DEBUG nova.network.neutron [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG nova.network.neutron [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating instance_info_cache with network_info: [{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Releasing lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG os_brick.utils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb19339c-3e05-4dce-8d70-897345eccc3f]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[afa87bfb-2b06-4f93-b3fd-fb9c26b8a786]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[23a241fb-c260-42d1-a2bb-8e68ac136080]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cccba57c-4ede-4221-9b97-5d001cf75141]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:25:42 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:25:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4cee00df-72ba-4b3c-9f10-310e138d372a]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:25:42 user nova-compute[71317]: DEBUG os_brick.utils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== get_connector_properties: return (122ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:25:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:44 user nova-compute[71317]: DEBUG nova.objects.instance [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'flavor' on Instance uuid 5273d202-ffc5-43c6-bb4f-f6d1a6855271 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.054s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:44 user nova-compute[71317]: INFO nova.compute.manager [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Attaching volume d8f57c49-602f-450f-8b9e-69f1f0255cee to /dev/vdb
Apr 14 04:25:44 user nova-compute[71317]: DEBUG os_brick.utils [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:25:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f8407f0-7f7c-49a1-a262-8643f8e1ed87]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[28fa44c7-889d-4864-931c-19becb13cc50]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a905976-1c47-407d-9156-9b9e5ef6b715]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc98f87d-3d92-4145-affe-2e09681246ff]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lSBjntEJ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:25:45 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:25:45 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:25:45 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:25:45 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[00f1499b-d4da-4a54-8643-74975ceddb09]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.utils [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] <== get_connector_properties: return (121ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updating existing volume attachment record: 9c3f0ff7-d8ee-47ff-b86d-78661c6187a1 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260" returned: 21 in 0.123s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[58dd5220-3354-4fa1-8bce-5abc6f139a8b]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:25:45 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.034s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3aa5979-5050-4616-87e3-2c0660481b31]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913] added
Apr 14 04:25:45 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d64d4321-3610-41c0-9501-24d4cacbec81]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e9f9130-17ee-433c-a37c-81db5c36dc3c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v lSBjntEJ {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v lSBjntEJ" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[74801c42-e728-45b0-b454-0aaf08c0eed2]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'lSBjntEJ'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[db298ec3-8707-4539-8fc3-aaeae1ffadef]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c14954f9-24f0-4d0b-ab09-be77d43472cc]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:45 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:45 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --login" returned: 0 in 0.071s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[350557cb-6d94-43a1-854f-16ca120eeb05]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913, portal: 172.16.0.220,3260]
Apr 14 04:25:45 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913, portal: 172.16.0.220,3260] successful.
Apr 14 04:25:45 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae1a8818-a30d-463c-9a8d-17ce07e713f2]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2555d5d4-bec8-4753-bff0-358c7c5e2ab6]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:45 user nova-compute[71317]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)
Apr 14 04:25:45 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:45 user nova-compute[71317]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)
Apr 14 04:25:45 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.414s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] HCTL ('34', '-', '-', 0) found on session 26 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab27ba2d-f983-4b43-b54e-cc42597f2753]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Searching for a device in session 26 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:25:45 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lSBjntEJ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:25:46 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.459s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:25:46 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] <== connect_volume: return (1462ms) {'type': 'block', 'scsi_wwn': '23539623166303836', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:25:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23539623166303836', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:25:46 user nova-compute[71317]: DEBUG nova.objects.instance [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'flavor' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:25:46 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:25:46 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:25:46 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:25:46 user nova-compute[71317]:   <serial>8ee7c552-e40e-4341-992f-722c82128913</serial>
Apr 14 04:25:46 user nova-compute[71317]: </disk>
Apr 14 04:25:46 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:25:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:25:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:25:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No VIF found with MAC fa:16:3e:17:59:61, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:25:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1f100ed4-bcb9-4964-abf9-733a0cd9df5c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.411s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Starting finish_migration {{(pid=71317) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11647}}
Apr 14 04:25:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance directory exists: not creating {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 04:25:47 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Creating image(s)
Apr 14 04:25:47 user nova-compute[71317]: DEBUG nova.objects.instance [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'trusted_certs' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.176s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Checking if we can resize image /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Cannot resize image /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Ensure instance console log exists: /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Start _get_guest_xml network_info=[{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:5d:8f:83"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': None, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MjAZDWGc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': 'a0a20aeb-ed06-4c65-b87d-74f8f825457d', 'attached_at': '2023-04-14T04:25:48.000000', 'detached_at': '', 'volume_id': '9cdf2c58-c295-483e-b766-13e01129ede2', 'serial': '9cdf2c58-c295-483e-b766-13e01129ede2'}, 'attachment_id': '31fe831d-8c6a-42ad-8f44-9b1909b9df01', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:25:48 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:25:48 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:57Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='84',id=12,is_public=True,memory_mb=192,name='m1.micro',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.objects.instance [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'vcpu_model' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MjAZDWGc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:25:48 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:25:48 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:25:48 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcab52e8-c5db-4bfe-a3dd-1ded4ce86b73]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:25:48 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[776cd76b-1d82-455f-9197-47bba1e40816]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2] added
Apr 14 04:25:48 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.118s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5a406c5-5141-4bb9-82f5-bae6097cd049]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[355aa247-322e-4b49-9e71-491c939ba228]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v MjAZDWGc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v MjAZDWGc" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a940be5d-7f1d-4acd-9cfe-7a5e62d86f48]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'MjAZDWGc'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c7ee011-bb0e-4355-9c22-fb50e2510246]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[47013769-dd16-4fb9-9d10-0c461e012c47]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:48 user nova-compute[71317]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)
Apr 14 04:25:48 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:48 user nova-compute[71317]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)
Apr 14 04:25:48 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --login" returned: 0 in 0.073s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[54d8ca36-ca69-4b1b-81ca-3e01391e28c2]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260]
Apr 14 04:25:48 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260] successful.
Apr 14 04:25:48 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[15cebd64-c962-4cdf-86e6-d27b11319d66]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a022193-e874-40ef-be9c-3cf201811307]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:48 user nova-compute[71317]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)
Apr 14 04:25:48 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:25:48 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:48 user nova-compute[71317]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)
Apr 14 04:25:48 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:25:48 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.506s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] HCTL ('35', '-', '-', 0) found on session 27 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d37e1c1-5f7f-4b50-bd45-a2b1ac895df7]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Searching for a device in session 27 and hctl ['35', '*', '*', 0] yield: None {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:25:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Searching for a device in session 27 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MjAZDWGc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.540s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== connect_volume: return (1541ms) {'type': 'block', 'scsi_wwn': '23661646666666433', 'path': '/dev/sdd'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23661646666666433', 'path': '/dev/sdd'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1350614351',display_name='tempest-ServerActionsTestJSON-server-1398117926',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1350614351',id=6,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:20:59Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-789bnjb3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:25:42Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=a0a20aeb-ed06-4c65-b87d-74f8f825457d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:5d:8f:83"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:5d:8f:83"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:25:49 user nova-compute[71317]:   <uuid>a0a20aeb-ed06-4c65-b87d-74f8f825457d</uuid>
Apr 14 04:25:49 user nova-compute[71317]:   <name>instance-00000006</name>
Apr 14 04:25:49 user nova-compute[71317]:   <memory>196608</memory>
Apr 14 04:25:49 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:25:49 user nova-compute[71317]:   <metadata>
Apr 14 04:25:49 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:25:49 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:25:49 user nova-compute[71317]:       <nova:name>tempest-ServerActionsTestJSON-server-1398117926</nova:name>
Apr 14 04:25:49 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:25:48</nova:creationTime>
Apr 14 04:25:49 user nova-compute[71317]:       <nova:flavor name="m1.micro">
Apr 14 04:25:49 user nova-compute[71317]:         <nova:memory>192</nova:memory>
Apr 14 04:25:49 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:25:49 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:25:49 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:25:49 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:25:49 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:25:49 user nova-compute[71317]:       <nova:owner>
Apr 14 04:25:49 user nova-compute[71317]:         <nova:user uuid="e35b3ae371cf4dd4b21e2999e58cd2f4">tempest-ServerActionsTestJSON-1048786641-project-member</nova:user>
Apr 14 04:25:49 user nova-compute[71317]:         <nova:project uuid="8c14c332dfcc44b49ddf3cb482dd0d4c">tempest-ServerActionsTestJSON-1048786641</nova:project>
Apr 14 04:25:49 user nova-compute[71317]:       </nova:owner>
Apr 14 04:25:49 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:25:49 user nova-compute[71317]:       <nova:ports>
Apr 14 04:25:49 user nova-compute[71317]:         <nova:port uuid="3a3e3b13-7b4e-46c8-9123-5c1f121a38b9">
Apr 14 04:25:49 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 04:25:49 user nova-compute[71317]:         </nova:port>
Apr 14 04:25:49 user nova-compute[71317]:       </nova:ports>
Apr 14 04:25:49 user nova-compute[71317]:     </nova:instance>
Apr 14 04:25:49 user nova-compute[71317]:   </metadata>
Apr 14 04:25:49 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:25:49 user nova-compute[71317]:     <system>
Apr 14 04:25:49 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:25:49 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:25:49 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:25:49 user nova-compute[71317]:       <entry name="serial">a0a20aeb-ed06-4c65-b87d-74f8f825457d</entry>
Apr 14 04:25:49 user nova-compute[71317]:       <entry name="uuid">a0a20aeb-ed06-4c65-b87d-74f8f825457d</entry>
Apr 14 04:25:49 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:25:49 user nova-compute[71317]:     </system>
Apr 14 04:25:49 user nova-compute[71317]:   </sysinfo>
Apr 14 04:25:49 user nova-compute[71317]:   <os>
Apr 14 04:25:49 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:25:49 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:25:49 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:25:49 user nova-compute[71317]:   </os>
Apr 14 04:25:49 user nova-compute[71317]:   <features>
Apr 14 04:25:49 user nova-compute[71317]:     <acpi/>
Apr 14 04:25:49 user nova-compute[71317]:     <apic/>
Apr 14 04:25:49 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:25:49 user nova-compute[71317]:   </features>
Apr 14 04:25:49 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:25:49 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:25:49 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:25:49 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:25:49 user nova-compute[71317]:   </clock>
Apr 14 04:25:49 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:25:49 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:25:49 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:25:49 user nova-compute[71317]:   </cpu>
Apr 14 04:25:49 user nova-compute[71317]:   <devices>
Apr 14 04:25:49 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:25:49 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:25:49 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk"/>
Apr 14 04:25:49 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:25:49 user nova-compute[71317]:     </disk>
Apr 14 04:25:49 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:25:49 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:25:49 user nova-compute[71317]:       <source dev="/dev/sdd"/>
Apr 14 04:25:49 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:25:49 user nova-compute[71317]:       <serial>9cdf2c58-c295-483e-b766-13e01129ede2</serial>
Apr 14 04:25:49 user nova-compute[71317]:     </disk>
Apr 14 04:25:49 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:25:49 user nova-compute[71317]:       <mac address="fa:16:3e:5d:8f:83"/>
Apr 14 04:25:49 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:25:49 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:25:49 user nova-compute[71317]:       <target dev="tap3a3e3b13-7b"/>
Apr 14 04:25:49 user nova-compute[71317]:     </interface>
Apr 14 04:25:49 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:25:49 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/console.log" append="off"/>
Apr 14 04:25:49 user nova-compute[71317]:     </serial>
Apr 14 04:25:49 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:25:49 user nova-compute[71317]:     <video>
Apr 14 04:25:49 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:25:49 user nova-compute[71317]:     </video>
Apr 14 04:25:49 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:25:49 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:25:49 user nova-compute[71317]:     </rng>
Apr 14 04:25:49 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:25:49 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:25:49 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:25:49 user nova-compute[71317]:     </memballoon>
Apr 14 04:25:49 user nova-compute[71317]:   </devices>
Apr 14 04:25:49 user nova-compute[71317]: </domain>
Apr 14 04:25:49 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1350614351',display_name='tempest-ServerActionsTestJSON-server-1398117926',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1350614351',id=6,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:20:59Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-789bnjb3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:25:42Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=a0a20aeb-ed06-4c65-b87d-74f8f825457d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:5d:8f:83"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:5d:8f:83"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG os_vif [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3a3e3b13-7b, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap3a3e3b13-7b, col_values=(('external_ids', {'iface-id': '3a3e3b13-7b4e-46c8-9123-5c1f121a38b9', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:5d:8f:83', 'vm-uuid': 'a0a20aeb-ed06-4c65-b87d-74f8f825457d'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:25:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:49 user nova-compute[71317]: INFO os_vif [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b')
Apr 14 04:25:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No VIF found with MAC fa:16:3e:5d:8f:83, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HfrLsjmV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:25:50 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:25:50 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:25:50 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4263e0c-11f2-4232-8df4-07d5691334b8]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:25:50 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c988227b-6af7-460c-ab4d-ce033ada1a6e]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee] added
Apr 14 04:25:50 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d4e9634-2f30-496b-ace6-0fc9234ce9f7]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9860aa91-b516-400c-a9e3-653458999295]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HfrLsjmV {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HfrLsjmV" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4a96631-56e3-4c78-be58-234733e80d4b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'HfrLsjmV'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f609661-1df3-4933-80f7-070afb640909]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f46df844-c791-4204-bcbe-5ab9b2190e63]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:50 user nova-compute[71317]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)
Apr 14 04:25:50 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:25:50 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:50 user nova-compute[71317]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)
Apr 14 04:25:50 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:25:50 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --login" returned: 0 in 0.184s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[21eb8456-a3f2-49d0-aeef-86393cd30f09]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee, portal: 172.16.0.220,3260]
Apr 14 04:25:50 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee, portal: 172.16.0.220,3260] successful.
Apr 14 04:25:50 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a848c7e1-7d18-43dd-a923-d0c41e9d8edd]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[115638b9-1ab4-4919-a435-f69070e71190]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:50 user nova-compute[71317]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)
Apr 14 04:25:50 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:25:50 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:25:50 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:25:50 user nova-compute[71317]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)
Apr 14 04:25:50 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:25:50 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:25:50 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.468s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] HCTL ('36', '-', '-', 0) found on session 28 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4484551-842c-4492-b542-dfdae48fe7e9]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Searching for a device in session 28 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:25:50 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HfrLsjmV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG nova.compute.manager [req-9d17a68d-7d26-4d84-a174-630a5d61e5b5 req-07767c19-d32e-4c0e-963d-f710b7458482 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9d17a68d-7d26-4d84-a174-630a5d61e5b5 req-07767c19-d32e-4c0e-963d-f710b7458482 service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9d17a68d-7d26-4d84-a174-630a5d61e5b5 req-07767c19-d32e-4c0e-963d-f710b7458482 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9d17a68d-7d26-4d84-a174-630a5d61e5b5 req-07767c19-d32e-4c0e-963d-f710b7458482 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG nova.compute.manager [req-9d17a68d-7d26-4d84-a174-630a5d61e5b5 req-07767c19-d32e-4c0e-963d-f710b7458482 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:25:51 user nova-compute[71317]: WARNING nova.compute.manager [req-9d17a68d-7d26-4d84-a174-630a5d61e5b5 req-07767c19-d32e-4c0e-963d-f710b7458482 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state active and task_state resize_finish.
Apr 14 04:25:51 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.508s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] <== connect_volume: return (1509ms) {'type': 'block', 'scsi_wwn': '23734376634363865', 'path': '/dev/sde'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23734376634363865', 'path': '/dev/sde'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG nova.objects.instance [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'flavor' on Instance uuid 5273d202-ffc5-43c6-bb4f-f6d1a6855271 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:25:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:25:51 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:25:51 user nova-compute[71317]:   <source dev="/dev/sde"/>
Apr 14 04:25:51 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:25:51 user nova-compute[71317]:   <serial>d8f57c49-602f-450f-8b9e-69f1f0255cee</serial>
Apr 14 04:25:51 user nova-compute[71317]: </disk>
Apr 14 04:25:51 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:25:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:25:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:25:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] No VIF found with MAC fa:16:3e:27:5d:35, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:25:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-821f1dd4-d2e1-4816-bea4-392d5e29a118 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.604s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-15966f3d-ea09-465f-b8fb-b9ba56ef2e87 req-cfe2dc5e-1356-47b9-8dfd-ddf2a8ccb508 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-15966f3d-ea09-465f-b8fb-b9ba56ef2e87 req-cfe2dc5e-1356-47b9-8dfd-ddf2a8ccb508 service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-15966f3d-ea09-465f-b8fb-b9ba56ef2e87 req-cfe2dc5e-1356-47b9-8dfd-ddf2a8ccb508 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-15966f3d-ea09-465f-b8fb-b9ba56ef2e87 req-cfe2dc5e-1356-47b9-8dfd-ddf2a8ccb508 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:25:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-15966f3d-ea09-465f-b8fb-b9ba56ef2e87 req-cfe2dc5e-1356-47b9-8dfd-ddf2a8ccb508 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:25:53 user nova-compute[71317]: WARNING nova.compute.manager [req-15966f3d-ea09-465f-b8fb-b9ba56ef2e87 req-cfe2dc5e-1356-47b9-8dfd-ddf2a8ccb508 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state active and task_state resize_finish.
Apr 14 04:25:53 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446338.897352, a0a20aeb-ed06-4c65-b87d-74f8f825457d => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:25:53 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] VM Stopped (Lifecycle Event)
Apr 14 04:25:53 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bb8178a8-a71a-40ab-bb99-bd52641a7256 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:25:54 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446354.0179708, a0a20aeb-ed06-4c65-b87d-74f8f825457d => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:25:54 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] VM Resumed (Lifecycle Event)
Apr 14 04:25:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:25:54 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance running successfully.
Apr 14 04:25:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Failed to set time: agent not configured {{(pid=71317) sync_guest_time /opt/stack/nova/nova/virt/libvirt/guest.py:200}}
Apr 14 04:25:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-73dfdb1e-0807-432c-bf28-6bfdb163a434 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] finish_migration finished successfully. {{(pid=71317) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11742}}
Apr 14 04:25:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:25:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:25:54 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] During sync_power_state the instance has a pending task (resize_finish). Skip.
Apr 14 04:25:54 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446354.018218, a0a20aeb-ed06-4c65-b87d-74f8f825457d => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:25:54 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] VM Started (Lifecycle Event)
Apr 14 04:25:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:25:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:25:54 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] During sync_power_state the instance has a pending task (resize_finish). Skip.
Apr 14 04:25:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-f3e9ce33-f9c4-48df-abb8-e3d97456d385 req-17b3db18-7005-4f19-b566-08ba99b97deb service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-changed-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-f3e9ce33-f9c4-48df-abb8-e3d97456d385 req-17b3db18-7005-4f19-b566-08ba99b97deb service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Refreshing instance network info cache due to event network-changed-dc662cd5-f74a-40ee-bc14-e0948cbc4f36. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:25:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f3e9ce33-f9c4-48df-abb8-e3d97456d385 req-17b3db18-7005-4f19-b566-08ba99b97deb service nova] Acquiring lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:25:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f3e9ce33-f9c4-48df-abb8-e3d97456d385 req-17b3db18-7005-4f19-b566-08ba99b97deb service nova] Acquired lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:25:55 user nova-compute[71317]: DEBUG nova.network.neutron [req-f3e9ce33-f9c4-48df-abb8-e3d97456d385 req-17b3db18-7005-4f19-b566-08ba99b97deb service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Refreshing network info cache for port dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:25:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-f3e9ce33-f9c4-48df-abb8-e3d97456d385 req-17b3db18-7005-4f19-b566-08ba99b97deb service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updated VIF entry in instance network info cache for port dc662cd5-f74a-40ee-bc14-e0948cbc4f36. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:25:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-f3e9ce33-f9c4-48df-abb8-e3d97456d385 req-17b3db18-7005-4f19-b566-08ba99b97deb service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updating instance_info_cache with network_info: [{"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:25:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f3e9ce33-f9c4-48df-abb8-e3d97456d385 req-17b3db18-7005-4f19-b566-08ba99b97deb service nova] Releasing lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:25:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-8f17bb6d-4c90-4111-9424-50e535cd34cc req-41e5af57-cc5f-44de-8e67-8c805219dfa1 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-changed-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-8f17bb6d-4c90-4111-9424-50e535cd34cc req-41e5af57-cc5f-44de-8e67-8c805219dfa1 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Refreshing instance network info cache due to event network-changed-dc662cd5-f74a-40ee-bc14-e0948cbc4f36. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8f17bb6d-4c90-4111-9424-50e535cd34cc req-41e5af57-cc5f-44de-8e67-8c805219dfa1 service nova] Acquiring lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8f17bb6d-4c90-4111-9424-50e535cd34cc req-41e5af57-cc5f-44de-8e67-8c805219dfa1 service nova] Acquired lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-8f17bb6d-4c90-4111-9424-50e535cd34cc req-41e5af57-cc5f-44de-8e67-8c805219dfa1 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Refreshing network info cache for port dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" acquired by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:25:59 user nova-compute[71317]: INFO nova.compute.manager [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Rebooting instance
Apr 14 04:25:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-8f17bb6d-4c90-4111-9424-50e535cd34cc req-41e5af57-cc5f-44de-8e67-8c805219dfa1 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updated VIF entry in instance network info cache for port dc662cd5-f74a-40ee-bc14-e0948cbc4f36. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-8f17bb6d-4c90-4111-9424-50e535cd34cc req-41e5af57-cc5f-44de-8e67-8c805219dfa1 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updating instance_info_cache with network_info: [{"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8f17bb6d-4c90-4111-9424-50e535cd34cc req-41e5af57-cc5f-44de-8e67-8c805219dfa1 service nova] Releasing lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquired lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG nova.network.neutron [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:25:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:00 user nova-compute[71317]: INFO nova.compute.manager [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Detaching volume 8ee7c552-e40e-4341-992f-722c82128913
Apr 14 04:26:00 user nova-compute[71317]: INFO nova.virt.block_device [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attempting to driver detach volume 8ee7c552-e40e-4341-992f-722c82128913 from mountpoint /dev/vdb
Apr 14 04:26:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Attempting to detach device vdb from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:26:00 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:26:00 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:26:00 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:26:00 user nova-compute[71317]:   <serial>8ee7c552-e40e-4341-992f-722c82128913</serial>
Apr 14 04:26:00 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:26:00 user nova-compute[71317]: </disk>
Apr 14 04:26:00 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:26:00 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Successfully detached device vdb from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the persistent domain config.
Apr 14 04:26:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:26:00 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:26:00 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:26:00 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:26:00 user nova-compute[71317]:   <serial>8ee7c552-e40e-4341-992f-722c82128913</serial>
Apr 14 04:26:00 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:26:00 user nova-compute[71317]: </disk>
Apr 14 04:26:00 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG nova.network.neutron [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updating instance_info_cache with network_info: [{"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Releasing lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446360.64288, f593ceb0-de44-490b-b365-3a05a3acd066 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:26:00 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Successfully detached device vdb from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the live domain config.
Apr 14 04:26:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lSBjntEJ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:26:00 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[35005e6b-4bb1-47a6-ac70-17e556dd694e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:26:00 user nova-compute[71317]: No targets found.
Apr 14 04:26:00 user nova-compute[71317]: iSNS:
Apr 14 04:26:00 user nova-compute[71317]: No targets found.
Apr 14 04:26:00 user nova-compute[71317]: STATIC:
Apr 14 04:26:00 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913
Apr 14 04:26:00 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:26:00 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:26:00 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee
Apr 14 04:26:00 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:26:00 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:26:00 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:26:00 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:26:00 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:26:00 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:26:00 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:26:00 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:26:00 user nova-compute[71317]: FIRMWARE:
Apr 14 04:26:00 user nova-compute[71317]: No targets found.
Apr 14 04:26:00 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:26:00 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:26:00 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b10bc5ed-7890-4f40-9c35-9c1e72489cca]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3a136bc-65be-466d-90a0-4dde7b93da7e]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:00 user nova-compute[71317]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)
Apr 14 04:26:00 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:00 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:00 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:00 user nova-compute[71317]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 (non-flash)
Apr 14 04:26:00 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:00 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:00 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d183fd90>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1ca0389-0580-4a3a-9b69-581692200240]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[da23953b-ae94-4158-b548-d47b62f32f07]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[82f712f7-17a1-4d9f-8f4a-6d6ddf3362f8]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.039s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f35fbe34-c91e-4d13-8ccf-b233ecab799e]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3446afb2-4f92-40f4-a638-4c4cb9efda7f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --logout" returned: 0 in 0.070s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6bf526a-e41a-4bac-b1ed-5b55335a9b22]: (4, ('Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913, portal: 172.16.0.220,3260]\nLogout of [sid: 26, target: iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913, portal: 172.16.0.220,3260]
Apr 14 04:26:00 user nova-compute[71317]: Logout of [sid: 26, target: iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913, portal: 172.16.0.220,3260] successful.
Apr 14 04:26:00 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ee7c552-e40e-4341-992f-722c82128913 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8556a03-48ae-4928-88a0-cc1673aefb62]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.376s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:26:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] <== disconnect_volume: return (378ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:26:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:26:01 user nova-compute[71317]: DEBUG nova.objects.instance [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'flavor' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-f46f2647-2e71-425b-b985-cefd73608fd8 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.898s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:26:02 user nova-compute[71317]: INFO nova.compute.claims [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Claim successful on node user
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:02 user nova-compute[71317]: INFO nova.compute.manager [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Terminating instance
Apr 14 04:26:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.682s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.306s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:26:03 user nova-compute[71317]: INFO nova.compute.claims [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Claim successful on node user
Apr 14 04:26:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:26:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:03 user nova-compute[71317]: INFO nova.virt.block_device [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Booting with volume snapshot 1a977934-32df-4abc-b8da-83e57587448a at /dev/vda
Apr 14 04:26:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.673s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:26:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.policy [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bcf79de7da27495dbfd14c556b5b54c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '693d95858c624feaa60114aa89082186', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.policy [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e4a70281d6734c15b3c4b1b00e4b5d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0848c9a67e2746e5b12b73920215b729', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Instance shutdown successfully.
Apr 14 04:26:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Instance destroyed successfully.
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.objects.instance [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'resources' on Instance uuid da3b4651-aaa0-430d-85b8-c15de386d08f {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:26:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:26:04 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Creating image(s)
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "/opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "/opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "/opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.005s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1781752187',display_name='tempest-AttachVolumeTestJSON-server-1781752187',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1781752187',id=3,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBY2UT8M9EXLRm2waCk4jzyD2YisXrBUU8nZo6LT6g0AlR0LbKytQJ1iJL1X5k+1hK+8YQxwsn1kgdkATbgdoU6lyDlwBLIIaqdqxbWOahpeOBX2u8LLHQwm2K/x2XVeAg==',key_name='tempest-keypair-1428467741',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:16:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6a89255339f74e55ad7c92967558bb81',ramdisk_id='',reservation_id='r-snsinryu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-991906288',owner_user_name='tempest-AttachVolumeTestJSON-991906288-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:24:17Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7610b45003e34f3395268977612f6c94',uuid=da3b4651-aaa0-430d-85b8-c15de386d08f,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converting VIF {"id": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "address": "fa:16:3e:68:ca:2f", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.192", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdf63af46-27", "ovs_interfaceid": "df63af46-27d1-4086-ae2c-39e8bc829bc7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG os_vif [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-909f8be4-22f7-4d94-a299-05c2dc23ebfc req-e23c807c-9e4d-4068-b722-3ef0909bd69d service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-vif-unplugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-909f8be4-22f7-4d94-a299-05c2dc23ebfc req-e23c807c-9e4d-4068-b722-3ef0909bd69d service nova] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-909f8be4-22f7-4d94-a299-05c2dc23ebfc req-e23c807c-9e4d-4068-b722-3ef0909bd69d service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-909f8be4-22f7-4d94-a299-05c2dc23ebfc req-e23c807c-9e4d-4068-b722-3ef0909bd69d service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-909f8be4-22f7-4d94-a299-05c2dc23ebfc req-e23c807c-9e4d-4068-b722-3ef0909bd69d service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] No waiting events found dispatching network-vif-unplugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:04 user nova-compute[71317]: WARNING nova.compute.manager [req-909f8be4-22f7-4d94-a299-05c2dc23ebfc req-e23c807c-9e4d-4068-b722-3ef0909bd69d service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received unexpected event network-vif-unplugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 for instance with vm_state active and task_state reboot_started.
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapdf63af46-27, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:04 user nova-compute[71317]: INFO os_vif [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:68:ca:2f,bridge_name='br-int',has_traffic_filtering=True,id=df63af46-27d1-4086-ae2c-39e8bc829bc7,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdf63af46-27')
Apr 14 04:26:04 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Deleting instance files /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f_del
Apr 14 04:26:04 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Deletion of /opt/stack/data/nova/instances/da3b4651-aaa0-430d-85b8-c15de386d08f_del complete
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.179s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:04 user nova-compute[71317]: INFO nova.compute.manager [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Took 1.43 seconds to destroy the instance on the hypervisor.
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.162s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk 1073741824" returned: 0 in 0.066s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.233s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.172s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Checking if we can resize image /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Cannot resize image /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lazy-loading 'migration_context' on Instance uuid 4a1c695b-2ca4-4652-8fb6-19accdab6ebe {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Ensure instance console log exists: /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Successfully created port: 260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:05 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Successfully created port: 8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:06 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Took 1.81 seconds to deallocate network for instance.
Apr 14 04:26:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd24d9cc-953a-4c74-9947-6d96e9c7a839 req-93cbcd9a-e7db-4566-9781-f58226513a27 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-deleted-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] No waiting events found dispatching network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:06 user nova-compute[71317]: WARNING nova.compute.manager [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received unexpected event network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 for instance with vm_state active and task_state reboot_started.
Apr 14 04:26:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-unplugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-unplugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-unplugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] Acquiring lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] No waiting events found dispatching network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:06 user nova-compute[71317]: WARNING nova.compute.manager [req-2373ad40-ad64-4421-9d44-40f3c2c98d34 req-d1c6421b-c1a4-45ec-9a47-e6cab56987f6 service nova] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Received unexpected event network-vif-plugged-df63af46-27d1-4086-ae2c-39e8bc829bc7 for instance with vm_state active and task_state deleting.
Apr 14 04:26:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:26:06 user nova-compute[71317]: INFO nova.compute.manager [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Rescuing
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquired lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:26:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.591s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:06 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Deleted allocations for instance da3b4651-aaa0-430d-85b8-c15de386d08f
Apr 14 04:26:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-563aadfb-098f-429c-a16c-bbf1ffa5f89a tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "da3b4651-aaa0-430d-85b8-c15de386d08f" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.245s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:07 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Successfully updated port: 260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:26:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "refresh_cache-4a1c695b-2ca4-4652-8fb6-19accdab6ebe" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:26:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquired lock "refresh_cache-4a1c695b-2ca4-4652-8fb6-19accdab6ebe" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:26:07 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:26:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-13459c7d-5414-49f5-a790-88941d5f8f3c req-5057d3fb-d4f9-4f98-a8a1-0353ac2965a2 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Received event network-changed-260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-13459c7d-5414-49f5-a790-88941d5f8f3c req-5057d3fb-d4f9-4f98-a8a1-0353ac2965a2 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Refreshing instance network info cache due to event network-changed-260596e7-554d-4e86-b146-fe487c284c83. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:26:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-13459c7d-5414-49f5-a790-88941d5f8f3c req-5057d3fb-d4f9-4f98-a8a1-0353ac2965a2 service nova] Acquiring lock "refresh_cache-4a1c695b-2ca4-4652-8fb6-19accdab6ebe" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:26:07 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:26:07 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updating instance_info_cache with network_info: [{"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Releasing lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:26:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.network.neutron [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Updating instance_info_cache with network_info: [{"id": "260596e7-554d-4e86-b146-fe487c284c83", "address": "fa:16:3e:5b:1b:7f", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap260596e7-55", "ovs_interfaceid": "260596e7-554d-4e86-b146-fe487c284c83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Releasing lock "refresh_cache-4a1c695b-2ca4-4652-8fb6-19accdab6ebe" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Instance network_info: |[{"id": "260596e7-554d-4e86-b146-fe487c284c83", "address": "fa:16:3e:5b:1b:7f", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap260596e7-55", "ovs_interfaceid": "260596e7-554d-4e86-b146-fe487c284c83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-13459c7d-5414-49f5-a790-88941d5f8f3c req-5057d3fb-d4f9-4f98-a8a1-0353ac2965a2 service nova] Acquired lock "refresh_cache-4a1c695b-2ca4-4652-8fb6-19accdab6ebe" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.network.neutron [req-13459c7d-5414-49f5-a790-88941d5f8f3c req-5057d3fb-d4f9-4f98-a8a1-0353ac2965a2 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Refreshing network info cache for port 260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Start _get_guest_xml network_info=[{"id": "260596e7-554d-4e86-b146-fe487c284c83", "address": "fa:16:3e:5b:1b:7f", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap260596e7-55", "ovs_interfaceid": "260596e7-554d-4e86-b146-fe487c284c83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:08 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:26:08 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:26:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-693983371',display_name='tempest-VolumesSnapshotTestJSON-instance-693983371',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-693983371',id=27,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0848c9a67e2746e5b12b73920215b729',ramdisk_id='',reservation_id='r-co7qa03q',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-1226436274',owner_user_name='tempest-VolumesSnapshotTestJSON-1226436274-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:26:04Z,user_data=None,user_id='e4a70281d6734c15b3c4b1b00e4b5d44',uuid=4a1c695b-2ca4-4652-8fb6-19accdab6ebe,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "260596e7-554d-4e86-b146-fe487c284c83", "address": "fa:16:3e:5b:1b:7f", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap260596e7-55", "ovs_interfaceid": "260596e7-554d-4e86-b146-fe487c284c83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Converting VIF {"id": "260596e7-554d-4e86-b146-fe487c284c83", "address": "fa:16:3e:5b:1b:7f", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap260596e7-55", "ovs_interfaceid": "260596e7-554d-4e86-b146-fe487c284c83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5b:1b:7f,bridge_name='br-int',has_traffic_filtering=True,id=260596e7-554d-4e86-b146-fe487c284c83,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap260596e7-55') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lazy-loading 'pci_devices' on Instance uuid 4a1c695b-2ca4-4652-8fb6-19accdab6ebe {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-6ffd597a-fe12-4d07-bcc3-1336ebeda63d req-bca4fcb9-11af-4f83-a4ea-3f1a8c5fea7d service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-unplugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6ffd597a-fe12-4d07-bcc3-1336ebeda63d req-bca4fcb9-11af-4f83-a4ea-3f1a8c5fea7d service nova] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6ffd597a-fe12-4d07-bcc3-1336ebeda63d req-bca4fcb9-11af-4f83-a4ea-3f1a8c5fea7d service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6ffd597a-fe12-4d07-bcc3-1336ebeda63d req-bca4fcb9-11af-4f83-a4ea-3f1a8c5fea7d service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-6ffd597a-fe12-4d07-bcc3-1336ebeda63d req-bca4fcb9-11af-4f83-a4ea-3f1a8c5fea7d service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] No waiting events found dispatching network-vif-unplugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:08 user nova-compute[71317]: WARNING nova.compute.manager [req-6ffd597a-fe12-4d07-bcc3-1336ebeda63d req-bca4fcb9-11af-4f83-a4ea-3f1a8c5fea7d service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received unexpected event network-vif-unplugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 for instance with vm_state active and task_state rescuing.
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for 5273d202-ffc5-43c6-bb4f-f6d1a6855271 due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446368.1765945, 5273d202-ffc5-43c6-bb4f-f6d1a6855271 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:26:08 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] VM Resumed (Lifecycle Event)
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:26:08 user nova-compute[71317]:   <uuid>4a1c695b-2ca4-4652-8fb6-19accdab6ebe</uuid>
Apr 14 04:26:08 user nova-compute[71317]:   <name>instance-0000001b</name>
Apr 14 04:26:08 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:26:08 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:26:08 user nova-compute[71317]:   <metadata>
Apr 14 04:26:08 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:26:08 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:26:08 user nova-compute[71317]:       <nova:name>tempest-VolumesSnapshotTestJSON-instance-693983371</nova:name>
Apr 14 04:26:08 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:26:08</nova:creationTime>
Apr 14 04:26:08 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:26:08 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:26:08 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:26:08 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:26:08 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:26:08 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:26:08 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:26:08 user nova-compute[71317]:       <nova:owner>
Apr 14 04:26:08 user nova-compute[71317]:         <nova:user uuid="e4a70281d6734c15b3c4b1b00e4b5d44">tempest-VolumesSnapshotTestJSON-1226436274-project-member</nova:user>
Apr 14 04:26:08 user nova-compute[71317]:         <nova:project uuid="0848c9a67e2746e5b12b73920215b729">tempest-VolumesSnapshotTestJSON-1226436274</nova:project>
Apr 14 04:26:08 user nova-compute[71317]:       </nova:owner>
Apr 14 04:26:08 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:26:08 user nova-compute[71317]:       <nova:ports>
Apr 14 04:26:08 user nova-compute[71317]:         <nova:port uuid="260596e7-554d-4e86-b146-fe487c284c83">
Apr 14 04:26:08 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 14 04:26:08 user nova-compute[71317]:         </nova:port>
Apr 14 04:26:08 user nova-compute[71317]:       </nova:ports>
Apr 14 04:26:08 user nova-compute[71317]:     </nova:instance>
Apr 14 04:26:08 user nova-compute[71317]:   </metadata>
Apr 14 04:26:08 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:26:08 user nova-compute[71317]:     <system>
Apr 14 04:26:08 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:26:08 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:26:08 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:26:08 user nova-compute[71317]:       <entry name="serial">4a1c695b-2ca4-4652-8fb6-19accdab6ebe</entry>
Apr 14 04:26:08 user nova-compute[71317]:       <entry name="uuid">4a1c695b-2ca4-4652-8fb6-19accdab6ebe</entry>
Apr 14 04:26:08 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:26:08 user nova-compute[71317]:     </system>
Apr 14 04:26:08 user nova-compute[71317]:   </sysinfo>
Apr 14 04:26:08 user nova-compute[71317]:   <os>
Apr 14 04:26:08 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:26:08 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:26:08 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:26:08 user nova-compute[71317]:   </os>
Apr 14 04:26:08 user nova-compute[71317]:   <features>
Apr 14 04:26:08 user nova-compute[71317]:     <acpi/>
Apr 14 04:26:08 user nova-compute[71317]:     <apic/>
Apr 14 04:26:08 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:26:08 user nova-compute[71317]:   </features>
Apr 14 04:26:08 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:26:08 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:26:08 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:26:08 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:26:08 user nova-compute[71317]:   </clock>
Apr 14 04:26:08 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:26:08 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:26:08 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:26:08 user nova-compute[71317]:   </cpu>
Apr 14 04:26:08 user nova-compute[71317]:   <devices>
Apr 14 04:26:08 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:26:08 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:26:08 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk"/>
Apr 14 04:26:08 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:26:08 user nova-compute[71317]:     </disk>
Apr 14 04:26:08 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:26:08 user nova-compute[71317]:       <mac address="fa:16:3e:5b:1b:7f"/>
Apr 14 04:26:08 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:26:08 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:26:08 user nova-compute[71317]:       <target dev="tap260596e7-55"/>
Apr 14 04:26:08 user nova-compute[71317]:     </interface>
Apr 14 04:26:08 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:26:08 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/console.log" append="off"/>
Apr 14 04:26:08 user nova-compute[71317]:     </serial>
Apr 14 04:26:08 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:26:08 user nova-compute[71317]:     <video>
Apr 14 04:26:08 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:26:08 user nova-compute[71317]:     </video>
Apr 14 04:26:08 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:26:08 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:26:08 user nova-compute[71317]:     </rng>
Apr 14 04:26:08 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:26:08 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:26:08 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:26:08 user nova-compute[71317]:     </memballoon>
Apr 14 04:26:08 user nova-compute[71317]:   </devices>
Apr 14 04:26:08 user nova-compute[71317]: </domain>
Apr 14 04:26:08 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:26:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-693983371',display_name='tempest-VolumesSnapshotTestJSON-instance-693983371',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-693983371',id=27,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0848c9a67e2746e5b12b73920215b729',ramdisk_id='',reservation_id='r-co7qa03q',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-1226436274',owner_user_name='tempest-VolumesSnapshotTestJSON-1226436274-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:26:04Z,user_data=None,user_id='e4a70281d6734c15b3c4b1b00e4b5d44',uuid=4a1c695b-2ca4-4652-8fb6-19accdab6ebe,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "260596e7-554d-4e86-b146-fe487c284c83", "address": "fa:16:3e:5b:1b:7f", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap260596e7-55", "ovs_interfaceid": "260596e7-554d-4e86-b146-fe487c284c83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Converting VIF {"id": "260596e7-554d-4e86-b146-fe487c284c83", "address": "fa:16:3e:5b:1b:7f", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap260596e7-55", "ovs_interfaceid": "260596e7-554d-4e86-b146-fe487c284c83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5b:1b:7f,bridge_name='br-int',has_traffic_filtering=True,id=260596e7-554d-4e86-b146-fe487c284c83,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap260596e7-55') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG os_vif [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5b:1b:7f,bridge_name='br-int',has_traffic_filtering=True,id=260596e7-554d-4e86-b146-fe487c284c83,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap260596e7-55') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-eba4816f-be78-4405-b976-adad70b496ec req-9f14d6b4-8054-4e71-b2ef-fe0c089ac88e service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-eba4816f-be78-4405-b976-adad70b496ec req-9f14d6b4-8054-4e71-b2ef-fe0c089ac88e service nova] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-eba4816f-be78-4405-b976-adad70b496ec req-9f14d6b4-8054-4e71-b2ef-fe0c089ac88e service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-eba4816f-be78-4405-b976-adad70b496ec req-9f14d6b4-8054-4e71-b2ef-fe0c089ac88e service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-eba4816f-be78-4405-b976-adad70b496ec req-9f14d6b4-8054-4e71-b2ef-fe0c089ac88e service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] No waiting events found dispatching network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:08 user nova-compute[71317]: WARNING nova.compute.manager [req-eba4816f-be78-4405-b976-adad70b496ec req-9f14d6b4-8054-4e71-b2ef-fe0c089ac88e service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received unexpected event network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 for instance with vm_state active and task_state reboot_started.
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-eba4816f-be78-4405-b976-adad70b496ec req-9f14d6b4-8054-4e71-b2ef-fe0c089ac88e service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-eba4816f-be78-4405-b976-adad70b496ec req-9f14d6b4-8054-4e71-b2ef-fe0c089ac88e service nova] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-eba4816f-be78-4405-b976-adad70b496ec req-9f14d6b4-8054-4e71-b2ef-fe0c089ac88e service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-eba4816f-be78-4405-b976-adad70b496ec req-9f14d6b4-8054-4e71-b2ef-fe0c089ac88e service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-eba4816f-be78-4405-b976-adad70b496ec req-9f14d6b4-8054-4e71-b2ef-fe0c089ac88e service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] No waiting events found dispatching network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:08 user nova-compute[71317]: WARNING nova.compute.manager [req-eba4816f-be78-4405-b976-adad70b496ec req-9f14d6b4-8054-4e71-b2ef-fe0c089ac88e service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received unexpected event network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 for instance with vm_state active and task_state reboot_started.
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap260596e7-55, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap260596e7-55, col_values=(('external_ids', {'iface-id': '260596e7-554d-4e86-b146-fe487c284c83', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:5b:1b:7f', 'vm-uuid': '4a1c695b-2ca4-4652-8fb6-19accdab6ebe'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:26:08 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Instance running successfully.
Apr 14 04:26:08 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Instance soft rebooted successfully.
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:08 user nova-compute[71317]: INFO os_vif [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5b:1b:7f,bridge_name='br-int',has_traffic_filtering=True,id=260596e7-554d-4e86-b146-fe487c284c83,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap260596e7-55')
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: reboot_started, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:26:08 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] During sync_power_state the instance has a pending task (reboot_started). Skip.
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446368.1774495, 5273d202-ffc5-43c6-bb4f-f6d1a6855271 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:26:08 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] VM Started (Lifecycle Event)
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: reboot_started, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3bda85c0-2ee6-49e1-9b97-a7506a3fa0b1 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" "released" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: held 8.585s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] No VIF found with MAC fa:16:3e:5b:1b:7f, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Successfully updated port: 8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "refresh_cache-aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquired lock "refresh_cache-aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:26:08 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Instance destroyed successfully.
Apr 14 04:26:08 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Attempting a stable device rescue
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=71317) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Instance directory exists: not creating {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 04:26:08 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Creating image(s)
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "/opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "/opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "/opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'trusted_certs' on Instance uuid 3df97db3-fd6c-4be6-af32-7cde9dbcce75 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "6f607dc5a06f489ce090518d55d9498f3378cadf" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "6f607dc5a06f489ce090518d55d9498f3378cadf" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.network.neutron [req-13459c7d-5414-49f5-a790-88941d5f8f3c req-5057d3fb-d4f9-4f98-a8a1-0353ac2965a2 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Updated VIF entry in instance network info cache for port 260596e7-554d-4e86-b146-fe487c284c83. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:26:08 user nova-compute[71317]: DEBUG nova.network.neutron [req-13459c7d-5414-49f5-a790-88941d5f8f3c req-5057d3fb-d4f9-4f98-a8a1-0353ac2965a2 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Updating instance_info_cache with network_info: [{"id": "260596e7-554d-4e86-b146-fe487c284c83", "address": "fa:16:3e:5b:1b:7f", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap260596e7-55", "ovs_interfaceid": "260596e7-554d-4e86-b146-fe487c284c83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-13459c7d-5414-49f5-a790-88941d5f8f3c req-5057d3fb-d4f9-4f98-a8a1-0353ac2965a2 service nova] Releasing lock "refresh_cache-4a1c695b-2ca4-4652-8fb6-19accdab6ebe" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf.part --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG os_brick.utils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c939620-cafb-4c78-a1d5-4f89a4b71364]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.035s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0090d751-e2f5-4ae8-b83a-7c6726baa9fb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ddd26592-bf33-4319-911a-8c0c0d89aa9d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ebb3d4e-0bcc-4a31-b882-d3689712bc8a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-0721d4e3-5244-48aa-bf26-afccd31a2e76 req-1e265db3-2acb-48b6-a4e6-989edea8b9ed service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0721d4e3-5244-48aa-bf26-afccd31a2e76 req-1e265db3-2acb-48b6-a4e6-989edea8b9ed service nova] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0721d4e3-5244-48aa-bf26-afccd31a2e76 req-1e265db3-2acb-48b6-a4e6-989edea8b9ed service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.006s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0721d4e3-5244-48aa-bf26-afccd31a2e76 req-1e265db3-2acb-48b6-a4e6-989edea8b9ed service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-0721d4e3-5244-48aa-bf26-afccd31a2e76 req-1e265db3-2acb-48b6-a4e6-989edea8b9ed service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] No waiting events found dispatching network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:10 user nova-compute[71317]: WARNING nova.compute.manager [req-0721d4e3-5244-48aa-bf26-afccd31a2e76 req-1e265db3-2acb-48b6-a4e6-989edea8b9ed service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received unexpected event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 for instance with vm_state active and task_state rescuing.
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf.part --force-share --output=json" returned: 0 in 0.228s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG nova.virt.images [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] 0d5722c7-7ae1-44d3-9599-66b2d7769656 was qcow2, converting to raw {{(pid=71317) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf.part /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf.converted {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-388729af-0f92-4631-9f6b-f6ddde126be5 req-69ac0a15-1b3c-416e-bc69-6ac38bf34981 service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Received event network-changed-8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-388729af-0f92-4631-9f6b-f6ddde126be5 req-69ac0a15-1b3c-416e-bc69-6ac38bf34981 service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Refreshing instance network info cache due to event network-changed-8864dbfd-6219-4843-aa68-aea87851f10b. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-388729af-0f92-4631-9f6b-f6ddde126be5 req-69ac0a15-1b3c-416e-bc69-6ac38bf34981 service nova] Acquiring lock "refresh_cache-aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Updating instance_info_cache with network_info: [{"id": "8864dbfd-6219-4843-aa68-aea87851f10b", "address": "fa:16:3e:f4:d4:39", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8864dbfd-62", "ovs_interfaceid": "8864dbfd-6219-4843-aa68-aea87851f10b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:26:10 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2f2cbe0-27d1-475a-a724-c43cfde24513]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG os_brick.utils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== get_connector_properties: return (241ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Updating existing volume attachment record: ea1f39d1-7b26-4825-bff6-cb389473233a {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Releasing lock "refresh_cache-aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Instance network_info: |[{"id": "8864dbfd-6219-4843-aa68-aea87851f10b", "address": "fa:16:3e:f4:d4:39", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8864dbfd-62", "ovs_interfaceid": "8864dbfd-6219-4843-aa68-aea87851f10b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-388729af-0f92-4631-9f6b-f6ddde126be5 req-69ac0a15-1b3c-416e-bc69-6ac38bf34981 service nova] Acquired lock "refresh_cache-aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG nova.network.neutron [req-388729af-0f92-4631-9f6b-f6ddde126be5 req-69ac0a15-1b3c-416e-bc69-6ac38bf34981 service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Refreshing network info cache for port 8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-2c8e6885-3630-41ce-8b7a-c0cbc292f92c req-cf771cf3-78e2-4217-9c8e-84bf3d6b2106 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Received event network-vif-plugged-260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2c8e6885-3630-41ce-8b7a-c0cbc292f92c req-cf771cf3-78e2-4217-9c8e-84bf3d6b2106 service nova] Acquiring lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2c8e6885-3630-41ce-8b7a-c0cbc292f92c req-cf771cf3-78e2-4217-9c8e-84bf3d6b2106 service nova] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2c8e6885-3630-41ce-8b7a-c0cbc292f92c req-cf771cf3-78e2-4217-9c8e-84bf3d6b2106 service nova] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-2c8e6885-3630-41ce-8b7a-c0cbc292f92c req-cf771cf3-78e2-4217-9c8e-84bf3d6b2106 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] No waiting events found dispatching network-vif-plugged-260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:10 user nova-compute[71317]: WARNING nova.compute.manager [req-2c8e6885-3630-41ce-8b7a-c0cbc292f92c req-cf771cf3-78e2-4217-9c8e-84bf3d6b2106 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Received unexpected event network-vif-plugged-260596e7-554d-4e86-b146-fe487c284c83 for instance with vm_state building and task_state spawning.
Apr 14 04:26:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf.part /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf.converted" returned: 0 in 0.517s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf.converted --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf.converted --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "6f607dc5a06f489ce090518d55d9498f3378cadf" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 2.093s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "6f607dc5a06f489ce090518d55d9498f3378cadf" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "6f607dc5a06f489ce090518d55d9498f3378cadf" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf,backing_fmt=raw /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.rescue {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.network.neutron [req-388729af-0f92-4631-9f6b-f6ddde126be5 req-69ac0a15-1b3c-416e-bc69-6ac38bf34981 service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Updated VIF entry in instance network info cache for port 8864dbfd-6219-4843-aa68-aea87851f10b. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.network.neutron [req-388729af-0f92-4631-9f6b-f6ddde126be5 req-69ac0a15-1b3c-416e-bc69-6ac38bf34981 service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Updating instance_info_cache with network_info: [{"id": "8864dbfd-6219-4843-aa68-aea87851f10b", "address": "fa:16:3e:f4:d4:39", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8864dbfd-62", "ovs_interfaceid": "8864dbfd-6219-4843-aa68-aea87851f10b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf,backing_fmt=raw /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.rescue" returned: 0 in 0.063s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "6f607dc5a06f489ce090518d55d9498f3378cadf" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.217s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'migration_context' on Instance uuid 3df97db3-fd6c-4be6-af32-7cde9dbcce75 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-388729af-0f92-4631-9f6b-f6ddde126be5 req-69ac0a15-1b3c-416e-bc69-6ac38bf34981 service nova] Releasing lock "refresh_cache-aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Start _get_guest_xml network_info=[{"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "vif_mac": "fa:16:3e:f6:fa:7c"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '0d5722c7-7ae1-44d3-9599-66b2d7769656', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YCyASkTZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, 'status': 'reserved', 'instance': '3df97db3-fd6c-4be6-af32-7cde9dbcce75', 'attached_at': '', 'detached_at': '', 'volume_id': '6e31b954-88d8-448b-bbb7-f19007a51866', 'serial': '6e31b954-88d8-448b-bbb7-f19007a51866'}, 'attachment_id': '2931a803-f7ab-4f6c-b752-edcd2987b0d0', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'resources' on Instance uuid 3df97db3-fd6c-4be6-af32-7cde9dbcce75 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'numa_topology' on Instance uuid 3df97db3-fd6c-4be6-af32-7cde9dbcce75 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:11 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:26:11 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'vcpu_model' on Instance uuid 3df97db3-fd6c-4be6-af32-7cde9dbcce75 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YCyASkTZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:26:11 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:26:11 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:26:11 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c73aec4-dddb-4891-8845-98d1518e37bc]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = YCyASkTZ\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Apr 14 04:26:11 user nova-compute[71317]: node.name = iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:26:11 user nova-compute[71317]: node.tpgt = -1
Apr 14 04:26:11 user nova-compute[71317]: node.startup = automatic
Apr 14 04:26:11 user nova-compute[71317]: node.leading_login = No
Apr 14 04:26:11 user nova-compute[71317]: iface.iscsi_ifacename = default
Apr 14 04:26:11 user nova-compute[71317]: iface.net_ifacename = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.ipaddress = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.prefix_len = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.hwaddress = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.transport_name = tcp
Apr 14 04:26:11 user nova-compute[71317]: iface.initiatorname = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.state = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.vlan_id = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.vlan_priority = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.vlan_state = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.iface_num = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.mtu = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.port = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.bootproto = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.subnet_mask = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.gateway = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.dhcp_alt_client_id_state = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.dhcp_alt_client_id = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.dhcp_dns = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.dhcp_learn_iqn = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.dhcp_req_vendor_id_state = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.dhcp_vendor_id_state = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.dhcp_vendor_id = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.dhcp_slp_da = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.fragmentation = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.gratuitous_arp = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.incoming_forwarding = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.tos_state = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.tos = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.ttl = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.delayed_ack = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.tcp_nagle = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.tcp_wsf_state = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.tcp_wsf = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.tcp_timer_scale = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.tcp_timestamp = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.redirect = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.def_task_mgmt_timeout = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.header_digest = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.data_digest = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.immediate_data = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.initial_r2t = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.data_seq_inorder = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.data_pdu_inorder = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.erl = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.max_receive_data_len = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.first_burst_len = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.max_outstanding_r2t = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.max_burst_len = 0
Apr 14 04:26:11 user nova-compute[71317]: iface.chap_auth = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.bidi_chap = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.strict_login_compliance = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.discovery_auth = <empty>
Apr 14 04:26:11 user nova-compute[71317]: iface.discovery_logout = <empty>
Apr 14 04:26:11 user nova-compute[71317]: node.discovery_address = <empty>
Apr 14 04:26:11 user nova-compute[71317]: node.discovery_port = 0
Apr 14 04:26:11 user nova-compute[71317]: node.discovery_type = static
Apr 14 04:26:11 user nova-compute[71317]: node.session.initial_cmdsn = 0
Apr 14 04:26:11 user nova-compute[71317]: node.session.initial_login_retry_max = 8
Apr 14 04:26:11 user nova-compute[71317]: node.session.xmit_thread_priority = -20
Apr 14 04:26:11 user nova-compute[71317]: node.session.cmds_max = 128
Apr 14 04:26:11 user nova-compute[71317]: node.session.queue_depth = 32
Apr 14 04:26:11 user nova-compute[71317]: node.session.nr_sessions = 1
Apr 14 04:26:11 user nova-compute[71317]: node.session.auth.authmethod = CHAP
Apr 14 04:26:11 user nova-compute[71317]: node.session.auth.username = YCyASkTZ
Apr 14 04:26:11 user nova-compute[71317]: node.session.auth.password = ***
Apr 14 04:26:11 user nova-compute[71317]: node.session.auth.username_in = <empty>
Apr 14 04:26:11 user nova-compute[71317]: node.session.auth.password_in = <empty>
Apr 14 04:26:11 user nova-compute[71317]: node.session.auth.chap_algs = MD5
Apr 14 04:26:11 user nova-compute[71317]: node.session.timeo.replacement_timeout = 120
Apr 14 04:26:11 user nova-compute[71317]: node.session.err_timeo.abort_timeout = 15
Apr 14 04:26:11 user nova-compute[71317]: node.session.err_timeo.lu_reset_timeout = 30
Apr 14 04:26:11 user nova-compute[71317]: node.session.err_timeo.tgt_reset_timeout = 30
Apr 14 04:26:11 user nova-compute[71317]: node.session.err_timeo.host_reset_timeout = 60
Apr 14 04:26:11 user nova-compute[71317]: node.session.iscsi.FastAbort = Yes
Apr 14 04:26:11 user nova-compute[71317]: node.session.iscsi.InitialR2T = No
Apr 14 04:26:11 user nova-compute[71317]: node.session.iscsi.ImmediateData = Yes
Apr 14 04:26:11 user nova-compute[71317]: node.session.iscsi.FirstBurstLength = 262144
Apr 14 04:26:11 user nova-compute[71317]: node.session.iscsi.MaxBurstLength = 16776192
Apr 14 04:26:11 user nova-compute[71317]: node.session.iscsi.DefaultTime2Retain = 0
Apr 14 04:26:11 user nova-compute[71317]: node.session.iscsi.DefaultTime2Wait = 2
Apr 14 04:26:11 user nova-compute[71317]: node.session.iscsi.MaxConnections = 1
Apr 14 04:26:11 user nova-compute[71317]: node.session.iscsi.MaxOutstandingR2T = 1
Apr 14 04:26:11 user nova-compute[71317]: node.session.iscsi.ERL = 0
Apr 14 04:26:11 user nova-compute[71317]: node.session.scan = manual
Apr 14 04:26:11 user nova-compute[71317]: node.session.reopen_max = 0
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].address = 172.16.0.220
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].port = 3260
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].startup = manual
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].tcp.window_size = 524288
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].tcp.type_of_service = 0
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].timeo.logout_timeout = 15
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].timeo.login_timeout = 15
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].timeo.auth_timeout = 45
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].timeo.noop_out_interval = 5
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].timeo.noop_out_timeout = 5
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].iscsi.HeaderDigest = None
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].iscsi.DataDigest = None
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].iscsi.IFMarker = No
Apr 14 04:26:11 user nova-compute[71317]: node.conn[0].iscsi.OFMarker = No
Apr 14 04:26:11 user nova-compute[71317]: # END RECORD
Apr 14 04:26:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[966ec16d-6191-4d5b-849e-a5fb1e07fc57]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[46a1c3af-eb4d-4bd4-b8ed-115604612d57]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YCyASkTZ {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YCyASkTZ" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[33c64dd9-f50c-43c7-8d95-1c26eacb487a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YCyASkTZ'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.070s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[06d35992-2ba3-4aa9-a85c-9629b5dc53ed]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[22722def-9dbf-4d5d-9181-1b3c49b1e1e5]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:11 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:11 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:11 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:11 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.226s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] HCTL ('33', '0', '0', 0) found on session 23 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Scanning host 33 c: 0, t: 0, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d851dbf-4f32-47a5-b9ca-62f5e99ebb53]: (4, ('0 0 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Searching for a device in session 23 and hctl ('33', '0', '0', 0) yield: sdb {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YCyASkTZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.263s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] <== connect_volume: return (265ms) {'type': 'block', 'scsi_wwn': '23466366334383665', 'path': '/dev/sdb'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23466366334383665', 'path': '/dev/sdb'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:24:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1117599705',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1117599705',id=24,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:24:31Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-5xp7dr1d',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:24:32Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=3df97db3-fd6c-4be6-af32-7cde9dbcce75,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "vif_mac": "fa:16:3e:f6:fa:7c"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "vif_mac": "fa:16:3e:f6:fa:7c"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f6:fa:7c,bridge_name='br-int',has_traffic_filtering=True,id=6baf0637-1ee9-4ccb-93a3-264041fb73f3,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6baf0637-1e') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'pci_devices' on Instance uuid 3df97db3-fd6c-4be6-af32-7cde9dbcce75 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:26:11 user nova-compute[71317]:   <uuid>3df97db3-fd6c-4be6-af32-7cde9dbcce75</uuid>
Apr 14 04:26:11 user nova-compute[71317]:   <name>instance-00000018</name>
Apr 14 04:26:11 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:26:11 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:26:11 user nova-compute[71317]:   <metadata>
Apr 14 04:26:11 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:26:11 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:26:11 user nova-compute[71317]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1117599705</nova:name>
Apr 14 04:26:11 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:26:11</nova:creationTime>
Apr 14 04:26:11 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:26:11 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:26:11 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:26:11 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:26:11 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:26:11 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:26:11 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:26:11 user nova-compute[71317]:       <nova:owner>
Apr 14 04:26:11 user nova-compute[71317]:         <nova:user uuid="6ec7103e06e643a6b9b2c908ba8d7b06">tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member</nova:user>
Apr 14 04:26:11 user nova-compute[71317]:         <nova:project uuid="d5995b934b0546c0983387e46e3b3df4">tempest-ServerBootFromVolumeStableRescueTest-247821842</nova:project>
Apr 14 04:26:11 user nova-compute[71317]:       </nova:owner>
Apr 14 04:26:11 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:26:11 user nova-compute[71317]:       <nova:ports>
Apr 14 04:26:11 user nova-compute[71317]:         <nova:port uuid="6baf0637-1ee9-4ccb-93a3-264041fb73f3">
Apr 14 04:26:11 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 14 04:26:11 user nova-compute[71317]:         </nova:port>
Apr 14 04:26:11 user nova-compute[71317]:       </nova:ports>
Apr 14 04:26:11 user nova-compute[71317]:     </nova:instance>
Apr 14 04:26:11 user nova-compute[71317]:   </metadata>
Apr 14 04:26:11 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:26:11 user nova-compute[71317]:     <system>
Apr 14 04:26:11 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:26:11 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:26:11 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:26:11 user nova-compute[71317]:       <entry name="serial">3df97db3-fd6c-4be6-af32-7cde9dbcce75</entry>
Apr 14 04:26:11 user nova-compute[71317]:       <entry name="uuid">3df97db3-fd6c-4be6-af32-7cde9dbcce75</entry>
Apr 14 04:26:11 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:26:11 user nova-compute[71317]:     </system>
Apr 14 04:26:11 user nova-compute[71317]:   </sysinfo>
Apr 14 04:26:11 user nova-compute[71317]:   <os>
Apr 14 04:26:11 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:26:11 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:26:11 user nova-compute[71317]:   </os>
Apr 14 04:26:11 user nova-compute[71317]:   <features>
Apr 14 04:26:11 user nova-compute[71317]:     <acpi/>
Apr 14 04:26:11 user nova-compute[71317]:     <apic/>
Apr 14 04:26:11 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:26:11 user nova-compute[71317]:   </features>
Apr 14 04:26:11 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:26:11 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:26:11 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:26:11 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:26:11 user nova-compute[71317]:   </clock>
Apr 14 04:26:11 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:26:11 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:26:11 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:26:11 user nova-compute[71317]:   </cpu>
Apr 14 04:26:11 user nova-compute[71317]:   <devices>
Apr 14 04:26:11 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:26:11 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:26:11 user nova-compute[71317]:       <source dev="/dev/sdb"/>
Apr 14 04:26:11 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:26:11 user nova-compute[71317]:       <serial>6e31b954-88d8-448b-bbb7-f19007a51866</serial>
Apr 14 04:26:11 user nova-compute[71317]:     </disk>
Apr 14 04:26:11 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:26:11 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:26:11 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.rescue"/>
Apr 14 04:26:11 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:26:11 user nova-compute[71317]:       <boot order="1"/>
Apr 14 04:26:11 user nova-compute[71317]:     </disk>
Apr 14 04:26:11 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:26:11 user nova-compute[71317]:       <mac address="fa:16:3e:f6:fa:7c"/>
Apr 14 04:26:11 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:26:11 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:26:11 user nova-compute[71317]:       <target dev="tap6baf0637-1e"/>
Apr 14 04:26:11 user nova-compute[71317]:     </interface>
Apr 14 04:26:11 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:26:11 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/console.log" append="off"/>
Apr 14 04:26:11 user nova-compute[71317]:     </serial>
Apr 14 04:26:11 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:26:11 user nova-compute[71317]:     <video>
Apr 14 04:26:11 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:26:11 user nova-compute[71317]:     </video>
Apr 14 04:26:11 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:26:11 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:26:11 user nova-compute[71317]:     </rng>
Apr 14 04:26:11 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:26:11 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:26:11 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:26:11 user nova-compute[71317]:     </memballoon>
Apr 14 04:26:11 user nova-compute[71317]:   </devices>
Apr 14 04:26:11 user nova-compute[71317]: </domain>
Apr 14 04:26:11 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:26:11 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Instance destroyed successfully.
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:26:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No VIF found with MAC fa:16:3e:f6:fa:7c, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-4bba5283-8683-442a-893e-e01345498253 req-ba61e0a2-8fde-4a2b-a669-180144ec7f9d service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Received event network-vif-plugged-260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4bba5283-8683-442a-893e-e01345498253 req-ba61e0a2-8fde-4a2b-a669-180144ec7f9d service nova] Acquiring lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4bba5283-8683-442a-893e-e01345498253 req-ba61e0a2-8fde-4a2b-a669-180144ec7f9d service nova] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4bba5283-8683-442a-893e-e01345498253 req-ba61e0a2-8fde-4a2b-a669-180144ec7f9d service nova] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-4bba5283-8683-442a-893e-e01345498253 req-ba61e0a2-8fde-4a2b-a669-180144ec7f9d service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] No waiting events found dispatching network-vif-plugged-260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:12 user nova-compute[71317]: WARNING nova.compute.manager [req-4bba5283-8683-442a-893e-e01345498253 req-ba61e0a2-8fde-4a2b-a669-180144ec7f9d service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Received unexpected event network-vif-plugged-260596e7-554d-4e86-b146-fe487c284c83 for instance with vm_state building and task_state spawning.
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446372.8891172, 4a1c695b-2ca4-4652-8fb6-19accdab6ebe => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:26:12 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] VM Resumed (Lifecycle Event)
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:26:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Instance spawned successfully.
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:12 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:26:12 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446372.8892887, 4a1c695b-2ca4-4652-8fb6-19accdab6ebe => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:26:12 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] VM Started (Lifecycle Event)
Apr 14 04:26:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:26:13 user nova-compute[71317]: INFO nova.compute.manager [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Took 9.06 seconds to spawn the instance on the hypervisor.
Apr 14 04:26:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:13 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:26:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:13 user nova-compute[71317]: INFO nova.compute.manager [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Took 10.57 seconds to build instance.
Apr 14 04:26:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef99ae98-bc9c-4308-a992-acf2217cf477 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.757s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-e8e083af-8fbe-43a9-9537-97da40ac5170 req-b333051a-b339-43d7-810f-dbdb2c03fbe8 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e8e083af-8fbe-43a9-9537-97da40ac5170 req-b333051a-b339-43d7-810f-dbdb2c03fbe8 service nova] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e8e083af-8fbe-43a9-9537-97da40ac5170 req-b333051a-b339-43d7-810f-dbdb2c03fbe8 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e8e083af-8fbe-43a9-9537-97da40ac5170 req-b333051a-b339-43d7-810f-dbdb2c03fbe8 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-e8e083af-8fbe-43a9-9537-97da40ac5170 req-b333051a-b339-43d7-810f-dbdb2c03fbe8 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] No waiting events found dispatching network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:13 user nova-compute[71317]: WARNING nova.compute.manager [req-e8e083af-8fbe-43a9-9537-97da40ac5170 req-b333051a-b339-43d7-810f-dbdb2c03fbe8 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received unexpected event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 for instance with vm_state active and task_state rescuing.
Apr 14 04:26:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-93a5a507-a874-459b-a428-43a6834fcda7 req-8e130252-70f4-4664-b043-973be00814f6 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-93a5a507-a874-459b-a428-43a6834fcda7 req-8e130252-70f4-4664-b043-973be00814f6 service nova] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-93a5a507-a874-459b-a428-43a6834fcda7 req-8e130252-70f4-4664-b043-973be00814f6 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-93a5a507-a874-459b-a428-43a6834fcda7 req-8e130252-70f4-4664-b043-973be00814f6 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-93a5a507-a874-459b-a428-43a6834fcda7 req-8e130252-70f4-4664-b043-973be00814f6 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] No waiting events found dispatching network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:15 user nova-compute[71317]: WARNING nova.compute.manager [req-93a5a507-a874-459b-a428-43a6834fcda7 req-8e130252-70f4-4664-b043-973be00814f6 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received unexpected event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 for instance with vm_state active and task_state rescuing.
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:26:16 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Creating image(s)
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Ensure instance console log exists: /opt/stack/data/nova/instances/aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Start _get_guest_xml network_info=[{"id": "8864dbfd-6219-4843-aa68-aea87851f10b", "address": "fa:16:3e:f4:d4:39", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8864dbfd-62", "ovs_interfaceid": "8864dbfd-6219-4843-aa68-aea87851f10b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': True, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PxcrKdyl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd', 'attached_at': '', 'detached_at': '', 'volume_id': '24291e20-0b7c-4b2c-9351-0e6cd072508f', 'serial': '24291e20-0b7c-4b2c-9351-0e6cd072508f'}, 'attachment_id': 'ea1f39d1-7b26-4825-bff6-cb389473233a', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:26:16 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:26:16 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PxcrKdyl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:26:16 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:26:16 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:26:16 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d987eab7-6a6d-42e4-a7eb-674565906b7f]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:26:16 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[269a5e37-5ce4-48e0-83ce-31dc1f9bf757]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f] added
Apr 14 04:26:16 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[03cd3f3c-796d-4603-98b2-3678da9253ec]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[150754e1-3228-48e6-95be-9472b3706979]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v PxcrKdyl {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v PxcrKdyl" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4193cc4a-848a-4bb8-aa47-e39960e7bafc]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'PxcrKdyl'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b3435d59-ebbe-4c83-b7ef-b699a68bfa8c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[69cf2ce7-2ea4-4b85-a550-14adcce18371]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:16 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:16 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:16 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:16 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:16 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:16 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for 3df97db3-fd6c-4be6-af32-7cde9dbcce75 due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446376.3303351, 3df97db3-fd6c-4be6-af32-7cde9dbcce75 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:26:16 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] VM Resumed (Lifecycle Event)
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2a49fde2-6a50-4f96-92f5-858e02e8f71c tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --login" returned: 0 in 0.103s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe519660-74b7-4fbd-920b-a43082155cbe]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f, portal: 172.16.0.220,3260]
Apr 14 04:26:16 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f, portal: 172.16.0.220,3260] successful.
Apr 14 04:26:16 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[828677e6-a270-4956-b4c9-3c2573d7527f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:16 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446376.3305154, 3df97db3-fd6c-4be6-af32-7cde9dbcce75 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:26:16 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] VM Started (Lifecycle Event)
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.059s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c8100d5-8e3e-4f55-8f42-6ad1582bacd8]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:16 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:16 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:16 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:26:16 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:16 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:16 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:16 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:26:16 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.405s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] HCTL ('34', '-', '-', 0) found on session 29 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4135979-30dd-44f1-8d42-7c4d1457a7c7]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Searching for a device in session 29 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:26:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PxcrKdyl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.445s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== connect_volume: return (1447ms) {'type': 'block', 'scsi_wwn': '26265613933373733', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26265613933373733', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:26:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1404624303',display_name='tempest-TestVolumeBootPattern-server-1404624303',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1404624303',id=26,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-oumzu04l',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:26:03Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8864dbfd-6219-4843-aa68-aea87851f10b", "address": "fa:16:3e:f4:d4:39", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8864dbfd-62", "ovs_interfaceid": "8864dbfd-6219-4843-aa68-aea87851f10b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "8864dbfd-6219-4843-aa68-aea87851f10b", "address": "fa:16:3e:f4:d4:39", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8864dbfd-62", "ovs_interfaceid": "8864dbfd-6219-4843-aa68-aea87851f10b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f4:d4:39,bridge_name='br-int',has_traffic_filtering=True,id=8864dbfd-6219-4843-aa68-aea87851f10b,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8864dbfd-62') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lazy-loading 'pci_devices' on Instance uuid aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:26:17 user nova-compute[71317]:   <uuid>aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd</uuid>
Apr 14 04:26:17 user nova-compute[71317]:   <name>instance-0000001a</name>
Apr 14 04:26:17 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:26:17 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:26:17 user nova-compute[71317]:   <metadata>
Apr 14 04:26:17 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:26:17 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:26:17 user nova-compute[71317]:       <nova:name>tempest-TestVolumeBootPattern-server-1404624303</nova:name>
Apr 14 04:26:17 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:26:16</nova:creationTime>
Apr 14 04:26:17 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:26:17 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:26:17 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:26:17 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:26:17 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:26:17 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:26:17 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:26:17 user nova-compute[71317]:       <nova:owner>
Apr 14 04:26:17 user nova-compute[71317]:         <nova:user uuid="bcf79de7da27495dbfd14c556b5b54c7">tempest-TestVolumeBootPattern-1780160917-project-member</nova:user>
Apr 14 04:26:17 user nova-compute[71317]:         <nova:project uuid="693d95858c624feaa60114aa89082186">tempest-TestVolumeBootPattern-1780160917</nova:project>
Apr 14 04:26:17 user nova-compute[71317]:       </nova:owner>
Apr 14 04:26:17 user nova-compute[71317]:       <nova:ports>
Apr 14 04:26:17 user nova-compute[71317]:         <nova:port uuid="8864dbfd-6219-4843-aa68-aea87851f10b">
Apr 14 04:26:17 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 04:26:17 user nova-compute[71317]:         </nova:port>
Apr 14 04:26:17 user nova-compute[71317]:       </nova:ports>
Apr 14 04:26:17 user nova-compute[71317]:     </nova:instance>
Apr 14 04:26:17 user nova-compute[71317]:   </metadata>
Apr 14 04:26:17 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:26:17 user nova-compute[71317]:     <system>
Apr 14 04:26:17 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:26:17 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:26:17 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:26:17 user nova-compute[71317]:       <entry name="serial">aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd</entry>
Apr 14 04:26:17 user nova-compute[71317]:       <entry name="uuid">aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd</entry>
Apr 14 04:26:17 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:26:17 user nova-compute[71317]:     </system>
Apr 14 04:26:17 user nova-compute[71317]:   </sysinfo>
Apr 14 04:26:17 user nova-compute[71317]:   <os>
Apr 14 04:26:17 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:26:17 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:26:17 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:26:17 user nova-compute[71317]:   </os>
Apr 14 04:26:17 user nova-compute[71317]:   <features>
Apr 14 04:26:17 user nova-compute[71317]:     <acpi/>
Apr 14 04:26:17 user nova-compute[71317]:     <apic/>
Apr 14 04:26:17 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:26:17 user nova-compute[71317]:   </features>
Apr 14 04:26:17 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:26:17 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:26:17 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:26:17 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:26:17 user nova-compute[71317]:   </clock>
Apr 14 04:26:17 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:26:17 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:26:17 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:26:17 user nova-compute[71317]:   </cpu>
Apr 14 04:26:17 user nova-compute[71317]:   <devices>
Apr 14 04:26:17 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:26:17 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:26:17 user nova-compute[71317]:       <source dev="/dev/sdc"/>
Apr 14 04:26:17 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:26:17 user nova-compute[71317]:       <serial>24291e20-0b7c-4b2c-9351-0e6cd072508f</serial>
Apr 14 04:26:17 user nova-compute[71317]:     </disk>
Apr 14 04:26:17 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:26:17 user nova-compute[71317]:       <mac address="fa:16:3e:f4:d4:39"/>
Apr 14 04:26:17 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:26:17 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:26:17 user nova-compute[71317]:       <target dev="tap8864dbfd-62"/>
Apr 14 04:26:17 user nova-compute[71317]:     </interface>
Apr 14 04:26:17 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:26:17 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd/console.log" append="off"/>
Apr 14 04:26:17 user nova-compute[71317]:     </serial>
Apr 14 04:26:17 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:26:17 user nova-compute[71317]:     <video>
Apr 14 04:26:17 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:26:17 user nova-compute[71317]:     </video>
Apr 14 04:26:17 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:26:17 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:26:17 user nova-compute[71317]:     </rng>
Apr 14 04:26:17 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:26:17 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:26:17 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:26:17 user nova-compute[71317]:     </memballoon>
Apr 14 04:26:17 user nova-compute[71317]:   </devices>
Apr 14 04:26:17 user nova-compute[71317]: </domain>
Apr 14 04:26:17 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:26:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1404624303',display_name='tempest-TestVolumeBootPattern-server-1404624303',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1404624303',id=26,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-oumzu04l',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:26:03Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8864dbfd-6219-4843-aa68-aea87851f10b", "address": "fa:16:3e:f4:d4:39", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8864dbfd-62", "ovs_interfaceid": "8864dbfd-6219-4843-aa68-aea87851f10b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "8864dbfd-6219-4843-aa68-aea87851f10b", "address": "fa:16:3e:f4:d4:39", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8864dbfd-62", "ovs_interfaceid": "8864dbfd-6219-4843-aa68-aea87851f10b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f4:d4:39,bridge_name='br-int',has_traffic_filtering=True,id=8864dbfd-6219-4843-aa68-aea87851f10b,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8864dbfd-62') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG os_vif [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f4:d4:39,bridge_name='br-int',has_traffic_filtering=True,id=8864dbfd-6219-4843-aa68-aea87851f10b,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8864dbfd-62') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8864dbfd-62, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap8864dbfd-62, col_values=(('external_ids', {'iface-id': '8864dbfd-6219-4843-aa68-aea87851f10b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f4:d4:39', 'vm-uuid': 'aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:17 user nova-compute[71317]: INFO os_vif [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f4:d4:39,bridge_name='br-int',has_traffic_filtering=True,id=8864dbfd-6219-4843-aa68-aea87851f10b,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8864dbfd-62')
Apr 14 04:26:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:26:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] No VIF found with MAC fa:16:3e:f4:d4:39, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:26:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:26:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:26:18 user nova-compute[71317]: INFO nova.compute.claims [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Claim successful on node user
Apr 14 04:26:18 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446363.92993, da3b4651-aaa0-430d-85b8-c15de386d08f => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:26:18 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] VM Stopped (Lifecycle Event)
Apr 14 04:26:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2b75b6e0-f093-41ba-b71b-95b4cba1f29a None None] [instance: da3b4651-aaa0-430d-85b8-c15de386d08f] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-631355a9-c92e-4945-8ab5-475df4314ea9 req-cee7bc26-9350-4dfc-b1b1-9e5e37ac091a service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Received event network-vif-plugged-8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-631355a9-c92e-4945-8ab5-475df4314ea9 req-cee7bc26-9350-4dfc-b1b1-9e5e37ac091a service nova] Acquiring lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-631355a9-c92e-4945-8ab5-475df4314ea9 req-cee7bc26-9350-4dfc-b1b1-9e5e37ac091a service nova] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.009s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-631355a9-c92e-4945-8ab5-475df4314ea9 req-cee7bc26-9350-4dfc-b1b1-9e5e37ac091a service nova] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-631355a9-c92e-4945-8ab5-475df4314ea9 req-cee7bc26-9350-4dfc-b1b1-9e5e37ac091a service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] No waiting events found dispatching network-vif-plugged-8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:19 user nova-compute[71317]: WARNING nova.compute.manager [req-631355a9-c92e-4945-8ab5-475df4314ea9 req-cee7bc26-9350-4dfc-b1b1-9e5e37ac091a service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Received unexpected event network-vif-plugged-8864dbfd-6219-4843-aa68-aea87851f10b for instance with vm_state building and task_state spawning.
Apr 14 04:26:19 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 1.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG nova.compute.manager [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG nova.compute.manager [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:26:19 user nova-compute[71317]: DEBUG nova.network.neutron [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:26:19 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:26:19 user nova-compute[71317]: DEBUG nova.compute.manager [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:26:19 user nova-compute[71317]: INFO nova.virt.block_device [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Booting with volume bcb3b184-a10d-482d-8c49-1bee42873be4 at /dev/vda
Apr 14 04:26:20 user nova-compute[71317]: DEBUG nova.policy [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '36b6981092424de781e8403946f956d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0e15d3d3e0943c4b6bd54abf2de5fee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG os_brick.utils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.036s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e42bd79-3548-4c61-be38-b6e73e724d42]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:20 user nova-compute[71317]: INFO nova.compute.manager [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Terminating instance
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b72d11ad-ae11-4637-98a0-50374c24e8ee]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3946a76b-9693-463e-ae5a-f7536d2f4925]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[46705910-f700-4ef3-81c8-160b61a60443]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:26:20 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3cada1f-c650-4a1d-82ba-762b20c2c45d]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG os_brick.utils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] <== get_connector_properties: return (232ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updating existing volume attachment record: bf5d96e9-8dbd-4ab4-97b5-3681b55419a4 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-cd2049eb-81f9-41c5-94fa-2dfdf342f374 req-251e5c83-aba2-4808-bdef-a3ab713831b9 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Received event network-vif-unplugged-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cd2049eb-81f9-41c5-94fa-2dfdf342f374 req-251e5c83-aba2-4808-bdef-a3ab713831b9 service nova] Acquiring lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cd2049eb-81f9-41c5-94fa-2dfdf342f374 req-251e5c83-aba2-4808-bdef-a3ab713831b9 service nova] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cd2049eb-81f9-41c5-94fa-2dfdf342f374 req-251e5c83-aba2-4808-bdef-a3ab713831b9 service nova] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-cd2049eb-81f9-41c5-94fa-2dfdf342f374 req-251e5c83-aba2-4808-bdef-a3ab713831b9 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] No waiting events found dispatching network-vif-unplugged-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-cd2049eb-81f9-41c5-94fa-2dfdf342f374 req-251e5c83-aba2-4808-bdef-a3ab713831b9 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Received event network-vif-unplugged-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:26:21 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Instance destroyed successfully.
Apr 14 04:26:21 user nova-compute[71317]: DEBUG nova.objects.instance [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lazy-loading 'resources' on Instance uuid d2e6cf32-080b-47b2-b4db-2b776ba85b38 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:20:31Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1275756548',display_name='tempest-VolumesAdminNegativeTest-server-1275756548',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1275756548',id=17,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCLbK9oP2daoZ7CXzxqtKBII/FQaajUVtxz7jBM0XYl4vW+UAcK//Iu4dDRXCklmwqWmqltHv3bFd4W3dO1IesZBh1iAMKvrzpnwuq9+FegSMWhSVQUKbRRtM51bR4jMkw==',key_name='tempest-keypair-1913633949',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:20:42Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3d389bb78d2e40b9b07d2d3ecdb7566b',ramdisk_id='',reservation_id='r-0cwwro3v',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesAdminNegativeTest-618964915',owner_user_name='tempest-VolumesAdminNegativeTest-618964915-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:20:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d2561bbc0c6d447da333ffdbb9fdc5f4',uuid=d2e6cf32-080b-47b2-b4db-2b776ba85b38,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "address": "fa:16:3e:ae:9a:66", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7c8689e1-9c", "ovs_interfaceid": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Converting VIF {"id": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "address": "fa:16:3e:ae:9a:66", "network": {"id": "617ed49c-d627-4bde-8573-e645f06f34b2", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1840235437-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3d389bb78d2e40b9b07d2d3ecdb7566b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7c8689e1-9c", "ovs_interfaceid": "7c8689e1-9c0a-42b6-8cb3-7751f619ba1f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ae:9a:66,bridge_name='br-int',has_traffic_filtering=True,id=7c8689e1-9c0a-42b6-8cb3-7751f619ba1f,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7c8689e1-9c') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG os_vif [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:ae:9a:66,bridge_name='br-int',has_traffic_filtering=True,id=7c8689e1-9c0a-42b6-8cb3-7751f619ba1f,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7c8689e1-9c') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7c8689e1-9c, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:26:21 user nova-compute[71317]: INFO os_vif [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:ae:9a:66,bridge_name='br-int',has_traffic_filtering=True,id=7c8689e1-9c0a-42b6-8cb3-7751f619ba1f,network=Network(617ed49c-d627-4bde-8573-e645f06f34b2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7c8689e1-9c')
Apr 14 04:26:21 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Deleting instance files /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38_del
Apr 14 04:26:21 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Deletion of /opt/stack/data/nova/instances/d2e6cf32-080b-47b2-b4db-2b776ba85b38_del complete
Apr 14 04:26:21 user nova-compute[71317]: INFO nova.compute.manager [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Took 1.04 seconds to destroy the instance on the hypervisor.
Apr 14 04:26:21 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-3821f7e2-7ab7-4ad5-bd73-261584146cae req-7caddcda-3a35-4421-b969-36a97eb3e358 service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Received event network-vif-plugged-8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3821f7e2-7ab7-4ad5-bd73-261584146cae req-7caddcda-3a35-4421-b969-36a97eb3e358 service nova] Acquiring lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3821f7e2-7ab7-4ad5-bd73-261584146cae req-7caddcda-3a35-4421-b969-36a97eb3e358 service nova] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3821f7e2-7ab7-4ad5-bd73-261584146cae req-7caddcda-3a35-4421-b969-36a97eb3e358 service nova] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-3821f7e2-7ab7-4ad5-bd73-261584146cae req-7caddcda-3a35-4421-b969-36a97eb3e358 service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] No waiting events found dispatching network-vif-plugged-8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:21 user nova-compute[71317]: WARNING nova.compute.manager [req-3821f7e2-7ab7-4ad5-bd73-261584146cae req-7caddcda-3a35-4421-b969-36a97eb3e358 service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Received unexpected event network-vif-plugged-8864dbfd-6219-4843-aa68-aea87851f10b for instance with vm_state building and task_state spawning.
Apr 14 04:26:21 user nova-compute[71317]: DEBUG nova.network.neutron [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Successfully created port: 61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446382.1293356, aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:26:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] VM Resumed (Lifecycle Event)
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:26:22 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Instance spawned successfully.
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:26:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446382.1378932, aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:26:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] VM Started (Lifecycle Event)
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:26:22 user nova-compute[71317]: INFO nova.compute.manager [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Took 6.28 seconds to spawn the instance on the hypervisor.
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:26:22 user nova-compute[71317]: INFO nova.compute.manager [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Took 20.22 seconds to build instance.
Apr 14 04:26:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1de54cae-2ff1-44b0-90e1-d06bf788dea6 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 20.335s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-f32b9271-4fcc-4504-8ae6-dfe55f0ef35d req-3f674fc1-8ebd-410d-8517-e68e6e3e587d service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Received event network-vif-deleted-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:22 user nova-compute[71317]: INFO nova.compute.manager [req-f32b9271-4fcc-4504-8ae6-dfe55f0ef35d req-3f674fc1-8ebd-410d-8517-e68e6e3e587d service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Neutron deleted interface 7c8689e1-9c0a-42b6-8cb3-7751f619ba1f; detaching it from the instance and deleting it from the info cache
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.network.neutron [req-f32b9271-4fcc-4504-8ae6-dfe55f0ef35d req-3f674fc1-8ebd-410d-8517-e68e6e3e587d service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:22 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Took 1.48 seconds to deallocate network for instance.
Apr 14 04:26:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-f32b9271-4fcc-4504-8ae6-dfe55f0ef35d req-3f674fc1-8ebd-410d-8517-e68e6e3e587d service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Detach interface failed, port_id=7c8689e1-9c0a-42b6-8cb3-7751f619ba1f, reason: Instance d2e6cf32-080b-47b2-b4db-2b776ba85b38 could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-b4e51ae7-0c83-4c05-9910-b29e03cbffd2 req-67ad50ca-1213-4ba5-83cb-7fe2423f53e1 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Received event network-vif-plugged-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b4e51ae7-0c83-4c05-9910-b29e03cbffd2 req-67ad50ca-1213-4ba5-83cb-7fe2423f53e1 service nova] Acquiring lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b4e51ae7-0c83-4c05-9910-b29e03cbffd2 req-67ad50ca-1213-4ba5-83cb-7fe2423f53e1 service nova] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b4e51ae7-0c83-4c05-9910-b29e03cbffd2 req-67ad50ca-1213-4ba5-83cb-7fe2423f53e1 service nova] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-b4e51ae7-0c83-4c05-9910-b29e03cbffd2 req-67ad50ca-1213-4ba5-83cb-7fe2423f53e1 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] No waiting events found dispatching network-vif-plugged-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:23 user nova-compute[71317]: WARNING nova.compute.manager [req-b4e51ae7-0c83-4c05-9910-b29e03cbffd2 req-67ad50ca-1213-4ba5-83cb-7fe2423f53e1 service nova] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Received unexpected event network-vif-plugged-7c8689e1-9c0a-42b6-8cb3-7751f619ba1f for instance with vm_state deleted and task_state None.
Apr 14 04:26:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Successfully updated port: 61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquired lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.619s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:26:23 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Deleted allocations for instance d2e6cf32-080b-47b2-b4db-2b776ba85b38
Apr 14 04:26:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1c5ee616-242f-4b51-a37c-5f4d75d5244b tempest-VolumesAdminNegativeTest-618964915 tempest-VolumesAdminNegativeTest-618964915-project-member] Lock "d2e6cf32-080b-47b2-b4db-2b776ba85b38" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.423s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updating instance_info_cache with network_info: [{"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Releasing lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:26:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Instance network_info: |[{"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:26:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-ad71c27b-2583-4d09-af94-676637cc0ae3 req-73530237-ce50-4081-b033-ffdd1f48a734 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-changed-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-ad71c27b-2583-4d09-af94-676637cc0ae3 req-73530237-ce50-4081-b033-ffdd1f48a734 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Refreshing instance network info cache due to event network-changed-61ec63cd-2733-40c4-a8c6-9243a6a3c144. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:26:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ad71c27b-2583-4d09-af94-676637cc0ae3 req-73530237-ce50-4081-b033-ffdd1f48a734 service nova] Acquiring lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:26:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ad71c27b-2583-4d09-af94-676637cc0ae3 req-73530237-ce50-4081-b033-ffdd1f48a734 service nova] Acquired lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:26:25 user nova-compute[71317]: DEBUG nova.network.neutron [req-ad71c27b-2583-4d09-af94-676637cc0ae3 req-73530237-ce50-4081-b033-ffdd1f48a734 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Refreshing network info cache for port 61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:26:25 user nova-compute[71317]: DEBUG nova.network.neutron [req-ad71c27b-2583-4d09-af94-676637cc0ae3 req-73530237-ce50-4081-b033-ffdd1f48a734 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updated VIF entry in instance network info cache for port 61ec63cd-2733-40c4-a8c6-9243a6a3c144. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:26:25 user nova-compute[71317]: DEBUG nova.network.neutron [req-ad71c27b-2583-4d09-af94-676637cc0ae3 req-73530237-ce50-4081-b033-ffdd1f48a734 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updating instance_info_cache with network_info: [{"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ad71c27b-2583-4d09-af94-676637cc0ae3 req-73530237-ce50-4081-b033-ffdd1f48a734 service nova] Releasing lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.compute.manager [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:26:26 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Creating image(s)
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Ensure instance console log exists: /opt/stack/data/nova/instances/6a510dd3-bb77-4f6d-bdfd-180ce1d31c71/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Start _get_guest_xml network_info=[{"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': True, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pRhHtNJB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '6a510dd3-bb77-4f6d-bdfd-180ce1d31c71', 'attached_at': '', 'detached_at': '', 'volume_id': 'bcb3b184-a10d-482d-8c49-1bee42873be4', 'serial': 'bcb3b184-a10d-482d-8c49-1bee42873be4'}, 'attachment_id': 'bf5d96e9-8dbd-4ab4-97b5-3681b55419a4', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:26:26 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:26:26 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pRhHtNJB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:26:26 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:26:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:26:26 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:26:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:26:26 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9bd29fd8-b4ab-4499-ba19-5893ef9f3334]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:26:26 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[db52a705-cf1d-4f1d-8bfb-d637175c66d2]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4] added
Apr 14 04:26:26 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2999dfa6-3326-4a60-bf54-3a3dead7a147]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[734c8e7a-8a29-4f35-a1ca-11a968c6a5cf]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pRhHtNJB {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pRhHtNJB" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3a6bc89-c3c3-4cf8-88db-d190a6a464a8]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'pRhHtNJB'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cafbab76-85f2-4001-8e64-d574eddf5e9e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9735e4ea-7c28-430a-8352-cce0186cbd8a]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:26:27 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:26:27 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --login" returned: 0 in 0.089s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd3c388e-9cc6-4f7d-a7c8-a5fa16360c06]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260]
Apr 14 04:26:27 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260] successful.
Apr 14 04:26:27 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.062s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d816b15-1f61-4c70-845f-7b1688d1476c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f883bb9-6565-4a7d-a4c5-e5b449a0e4ef]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:26:27 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:26:27 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:26:27 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.390s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] HCTL ('37', '-', '-', 0) found on session 30 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7980835-a5ff-4cb5-bb00-4d3fae9de4d1]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Searching for a device in session 30 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:26:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pRhHtNJB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.469s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] <== connect_volume: return (1471ms) {'type': 'block', 'scsi_wwn': '23164336261353930', 'path': '/dev/sdf'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23164336261353930', 'path': '/dev/sdf'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:26:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestShelveInstance-server-2065386753',display_name='tempest-TestShelveInstance-server-2065386753',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-2065386753',id=28,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGGk9hQ8wpWxQD87wrkXLIqSgBhES95isERnHNc/wFOMwogmDvk0s+HHEuuYbkzB2pEyv6u9qIhQ2bQkxjPV0WSr048nFyVVwxTOQQiEcmHX8pNahIhzR1Wk0Uv6MFb63A==',key_name='tempest-TestShelveInstance-2060150193',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='b0e15d3d3e0943c4b6bd54abf2de5fee',ramdisk_id='',reservation_id='r-osj4jtne',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestShelveInstance-1458140851',owner_user_name='tempest-TestShelveInstance-1458140851-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:26:20Z,user_data=None,user_id='36b6981092424de781e8403946f956d3',uuid=6a510dd3-bb77-4f6d-bdfd-180ce1d31c71,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Converting VIF {"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG nova.objects.instance [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lazy-loading 'pci_devices' on Instance uuid 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:26:28 user nova-compute[71317]:   <uuid>6a510dd3-bb77-4f6d-bdfd-180ce1d31c71</uuid>
Apr 14 04:26:28 user nova-compute[71317]:   <name>instance-0000001c</name>
Apr 14 04:26:28 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:26:28 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:26:28 user nova-compute[71317]:   <metadata>
Apr 14 04:26:28 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:26:28 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:26:28 user nova-compute[71317]:       <nova:name>tempest-TestShelveInstance-server-2065386753</nova:name>
Apr 14 04:26:28 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:26:26</nova:creationTime>
Apr 14 04:26:28 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:26:28 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:26:28 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:26:28 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:26:28 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:26:28 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:26:28 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:26:28 user nova-compute[71317]:       <nova:owner>
Apr 14 04:26:28 user nova-compute[71317]:         <nova:user uuid="36b6981092424de781e8403946f956d3">tempest-TestShelveInstance-1458140851-project-member</nova:user>
Apr 14 04:26:28 user nova-compute[71317]:         <nova:project uuid="b0e15d3d3e0943c4b6bd54abf2de5fee">tempest-TestShelveInstance-1458140851</nova:project>
Apr 14 04:26:28 user nova-compute[71317]:       </nova:owner>
Apr 14 04:26:28 user nova-compute[71317]:       <nova:ports>
Apr 14 04:26:28 user nova-compute[71317]:         <nova:port uuid="61ec63cd-2733-40c4-a8c6-9243a6a3c144">
Apr 14 04:26:28 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 14 04:26:28 user nova-compute[71317]:         </nova:port>
Apr 14 04:26:28 user nova-compute[71317]:       </nova:ports>
Apr 14 04:26:28 user nova-compute[71317]:     </nova:instance>
Apr 14 04:26:28 user nova-compute[71317]:   </metadata>
Apr 14 04:26:28 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:26:28 user nova-compute[71317]:     <system>
Apr 14 04:26:28 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:26:28 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:26:28 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:26:28 user nova-compute[71317]:       <entry name="serial">6a510dd3-bb77-4f6d-bdfd-180ce1d31c71</entry>
Apr 14 04:26:28 user nova-compute[71317]:       <entry name="uuid">6a510dd3-bb77-4f6d-bdfd-180ce1d31c71</entry>
Apr 14 04:26:28 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:26:28 user nova-compute[71317]:     </system>
Apr 14 04:26:28 user nova-compute[71317]:   </sysinfo>
Apr 14 04:26:28 user nova-compute[71317]:   <os>
Apr 14 04:26:28 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:26:28 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:26:28 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:26:28 user nova-compute[71317]:   </os>
Apr 14 04:26:28 user nova-compute[71317]:   <features>
Apr 14 04:26:28 user nova-compute[71317]:     <acpi/>
Apr 14 04:26:28 user nova-compute[71317]:     <apic/>
Apr 14 04:26:28 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:26:28 user nova-compute[71317]:   </features>
Apr 14 04:26:28 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:26:28 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:26:28 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:26:28 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:26:28 user nova-compute[71317]:   </clock>
Apr 14 04:26:28 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:26:28 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:26:28 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:26:28 user nova-compute[71317]:   </cpu>
Apr 14 04:26:28 user nova-compute[71317]:   <devices>
Apr 14 04:26:28 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:26:28 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:26:28 user nova-compute[71317]:       <source dev="/dev/sdf"/>
Apr 14 04:26:28 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:26:28 user nova-compute[71317]:       <serial>bcb3b184-a10d-482d-8c49-1bee42873be4</serial>
Apr 14 04:26:28 user nova-compute[71317]:     </disk>
Apr 14 04:26:28 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:26:28 user nova-compute[71317]:       <mac address="fa:16:3e:fb:2e:cd"/>
Apr 14 04:26:28 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:26:28 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:26:28 user nova-compute[71317]:       <target dev="tap61ec63cd-27"/>
Apr 14 04:26:28 user nova-compute[71317]:     </interface>
Apr 14 04:26:28 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:26:28 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/6a510dd3-bb77-4f6d-bdfd-180ce1d31c71/console.log" append="off"/>
Apr 14 04:26:28 user nova-compute[71317]:     </serial>
Apr 14 04:26:28 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:26:28 user nova-compute[71317]:     <video>
Apr 14 04:26:28 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:26:28 user nova-compute[71317]:     </video>
Apr 14 04:26:28 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:26:28 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:26:28 user nova-compute[71317]:     </rng>
Apr 14 04:26:28 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:26:28 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:26:28 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:26:28 user nova-compute[71317]:     </memballoon>
Apr 14 04:26:28 user nova-compute[71317]:   </devices>
Apr 14 04:26:28 user nova-compute[71317]: </domain>
Apr 14 04:26:28 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:26:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestShelveInstance-server-2065386753',display_name='tempest-TestShelveInstance-server-2065386753',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-2065386753',id=28,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGGk9hQ8wpWxQD87wrkXLIqSgBhES95isERnHNc/wFOMwogmDvk0s+HHEuuYbkzB2pEyv6u9qIhQ2bQkxjPV0WSr048nFyVVwxTOQQiEcmHX8pNahIhzR1Wk0Uv6MFb63A==',key_name='tempest-TestShelveInstance-2060150193',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='b0e15d3d3e0943c4b6bd54abf2de5fee',ramdisk_id='',reservation_id='r-osj4jtne',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestShelveInstance-1458140851',owner_user_name='tempest-TestShelveInstance-1458140851-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:26:20Z,user_data=None,user_id='36b6981092424de781e8403946f956d3',uuid=6a510dd3-bb77-4f6d-bdfd-180ce1d31c71,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Converting VIF {"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG os_vif [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap61ec63cd-27, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap61ec63cd-27, col_values=(('external_ids', {'iface-id': '61ec63cd-2733-40c4-a8c6-9243a6a3c144', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:fb:2e:cd', 'vm-uuid': '6a510dd3-bb77-4f6d-bdfd-180ce1d31c71'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:28 user nova-compute[71317]: INFO os_vif [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27')
Apr 14 04:26:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:26:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] No VIF found with MAC fa:16:3e:fb:2e:cd, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:26:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:30 user nova-compute[71317]: DEBUG nova.compute.manager [req-4e7f0cca-3efb-423a-8e8e-beeaee699caa req-23230edd-b3f2-40ae-8c49-4d110abba92c service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4e7f0cca-3efb-423a-8e8e-beeaee699caa req-23230edd-b3f2-40ae-8c49-4d110abba92c service nova] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4e7f0cca-3efb-423a-8e8e-beeaee699caa req-23230edd-b3f2-40ae-8c49-4d110abba92c service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4e7f0cca-3efb-423a-8e8e-beeaee699caa req-23230edd-b3f2-40ae-8c49-4d110abba92c service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:30 user nova-compute[71317]: DEBUG nova.compute.manager [req-4e7f0cca-3efb-423a-8e8e-beeaee699caa req-23230edd-b3f2-40ae-8c49-4d110abba92c service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] No waiting events found dispatching network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:30 user nova-compute[71317]: WARNING nova.compute.manager [req-4e7f0cca-3efb-423a-8e8e-beeaee699caa req-23230edd-b3f2-40ae-8c49-4d110abba92c service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received unexpected event network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 for instance with vm_state building and task_state spawning.
Apr 14 04:26:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:31 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:26:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:26:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:26:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:26:31 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446392.215526, 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:26:32 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] VM Resumed (Lifecycle Event)
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:26:32 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Instance spawned successfully.
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:26:32 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446392.2168283, 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:26:32 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] VM Started (Lifecycle Event)
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-b7aa4ae1-6c2c-4c3c-a386-266c28220c6a req-60d1df1d-2004-4424-8599-882d992dfa33 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b7aa4ae1-6c2c-4c3c-a386-266c28220c6a req-60d1df1d-2004-4424-8599-882d992dfa33 service nova] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b7aa4ae1-6c2c-4c3c-a386-266c28220c6a req-60d1df1d-2004-4424-8599-882d992dfa33 service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b7aa4ae1-6c2c-4c3c-a386-266c28220c6a req-60d1df1d-2004-4424-8599-882d992dfa33 service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-b7aa4ae1-6c2c-4c3c-a386-266c28220c6a req-60d1df1d-2004-4424-8599-882d992dfa33 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] No waiting events found dispatching network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:26:32 user nova-compute[71317]: WARNING nova.compute.manager [req-b7aa4ae1-6c2c-4c3c-a386-266c28220c6a req-60d1df1d-2004-4424-8599-882d992dfa33 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received unexpected event network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 for instance with vm_state building and task_state spawning.
Apr 14 04:26:32 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Updating instance_info_cache with network_info: [{"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:32 user nova-compute[71317]: INFO nova.compute.manager [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Took 5.60 seconds to spawn the instance on the hypervisor.
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:26:32 user nova-compute[71317]: INFO nova.compute.manager [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Took 13.89 seconds to build instance.
Apr 14 04:26:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-975c9fcd-0b03-4103-afc9-56d439ada8a6 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 14.025s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:26:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:33 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:26:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.178s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.173s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446381.015669, d2e6cf32-080b-47b2-b4db-2b776ba85b38 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:26:36 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] VM Stopped (Lifecycle Event)
Apr 14 04:26:36 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a74e8221-1f6e-4587-87c7-5576968953d0 None None] [instance: d2e6cf32-080b-47b2-b4db-2b776ba85b38] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:26:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.rescue --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.rescue --force-share --output=json" returned: 0 in 0.158s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:26:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:38 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:26:38 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7236MB free_disk=26.2747802734375GB free_vcpus=1 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Applying migration context for instance a0a20aeb-ed06-4c65-b87d-74f8f825457d as it has an incoming, in-progress migration c4903c05-a131-4f02-b6b6-805ce86d54cd. Migration status is finished {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:950}}
Apr 14 04:26:38 user nova-compute[71317]: INFO nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating resource usage from migration c4903c05-a131-4f02-b6b6-805ce86d54cd
Apr 14 04:26:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance f593ceb0-de44-490b-b365-3a05a3acd066 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 5273d202-ffc5-43c6-bb4f-f6d1a6855271 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Migration c4903c05-a131-4f02-b6b6-805ce86d54cd is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1637}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 4a1c695b-2ca4-4652-8fb6-19accdab6ebe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 12 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=2112MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=12 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:26:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.502s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'flavor' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.059s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:54 user nova-compute[71317]: INFO nova.compute.manager [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attaching volume 430bf7a7-6067-4516-9a60-f895bded451d to /dev/vdb
Apr 14 04:26:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG os_brick.utils [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[adb796a4-a2e6-4f36-bf4c-257b18453e7a]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd5d11af-f6d6-460b-8995-d4fd3c36b189]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[83899f35-328a-40c5-a0cd-3e9895cb0e33]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[96d34976-08ec-4548-97e8-5154505383cd]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:26:54 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:26:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cfbbeebf-1f86-4c91-8df9-a6ee469db9d9]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG os_brick.utils [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] <== get_connector_properties: return (126ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:26:54 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Updating existing volume attachment record: b17d6cbb-c3e4-4370-8725-f0aa44af34d9 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:26:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:26:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:56 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:26:56 user nova-compute[71317]: INFO nova.compute.claims [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Claim successful on node user
Apr 14 04:26:57 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.477s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG nova.compute.manager [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG nova.compute.manager [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG nova.network.neutron [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:26:57 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:26:57 user nova-compute[71317]: DEBUG nova.compute.manager [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG nova.policy [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7610b45003e34f3395268977612f6c94', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a89255339f74e55ad7c92967558bb81', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG nova.compute.manager [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:26:57 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Creating image(s)
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "/opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "/opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "/opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk 1073741824" returned: 0 in 0.050s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.194s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Checking if we can resize image /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Cannot resize image /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG nova.objects.instance [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'migration_context' on Instance uuid e19eb131-ae90-4220-9d47-15e8b8b332de {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Ensure instance console log exists: /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Successfully created port: fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Successfully updated port: fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "refresh_cache-e19eb131-ae90-4220-9d47-15e8b8b332de" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquired lock "refresh_cache-e19eb131-ae90-4220-9d47-15e8b8b332de" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:26:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-456ecbd1-5e8c-4aa5-8e23-62f208e174f4 req-ecff8947-5518-41fe-9392-cbcab6490f58 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Received event network-changed-fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-456ecbd1-5e8c-4aa5-8e23-62f208e174f4 req-ecff8947-5518-41fe-9392-cbcab6490f58 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Refreshing instance network info cache due to event network-changed-fab305ad-6649-4e13-944b-82a2dbc76547. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-456ecbd1-5e8c-4aa5-8e23-62f208e174f4 req-ecff8947-5518-41fe-9392-cbcab6490f58 service nova] Acquiring lock "refresh_cache-e19eb131-ae90-4220-9d47-15e8b8b332de" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.network.neutron [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Updating instance_info_cache with network_info: [{"id": "fab305ad-6649-4e13-944b-82a2dbc76547", "address": "fa:16:3e:ec:97:fb", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfab305ad-66", "ovs_interfaceid": "fab305ad-6649-4e13-944b-82a2dbc76547", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Releasing lock "refresh_cache-e19eb131-ae90-4220-9d47-15e8b8b332de" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.compute.manager [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Instance network_info: |[{"id": "fab305ad-6649-4e13-944b-82a2dbc76547", "address": "fa:16:3e:ec:97:fb", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfab305ad-66", "ovs_interfaceid": "fab305ad-6649-4e13-944b-82a2dbc76547", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-456ecbd1-5e8c-4aa5-8e23-62f208e174f4 req-ecff8947-5518-41fe-9392-cbcab6490f58 service nova] Acquired lock "refresh_cache-e19eb131-ae90-4220-9d47-15e8b8b332de" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-456ecbd1-5e8c-4aa5-8e23-62f208e174f4 req-ecff8947-5518-41fe-9392-cbcab6490f58 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Refreshing network info cache for port fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Start _get_guest_xml network_info=[{"id": "fab305ad-6649-4e13-944b-82a2dbc76547", "address": "fa:16:3e:ec:97:fb", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfab305ad-66", "ovs_interfaceid": "fab305ad-6649-4e13-944b-82a2dbc76547", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'xaQKgXWG', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:26:59 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:26:59 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:26:59 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:59 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:26:59 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:26:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1852573542',display_name='tempest-AttachVolumeTestJSON-server-1852573542',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1852573542',id=29,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPahbjwxSvxq0IxMz5/M3Sbn33mEC8YBDkFVpCOH/WAboRbjSbxsllitzmJB7SVS4TKY3MCm6q57KW3T5WFQQOYggXTJylYDmiHUt2/y+RsASQ2RPX3ud6TI2ErXR0uk4g==',key_name='tempest-keypair-1690869240',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6a89255339f74e55ad7c92967558bb81',ramdisk_id='',reservation_id='r-j2gf6jl3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-991906288',owner_user_name='tempest-AttachVolumeTestJSON-991906288-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:26:57Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7610b45003e34f3395268977612f6c94',uuid=e19eb131-ae90-4220-9d47-15e8b8b332de,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fab305ad-6649-4e13-944b-82a2dbc76547", "address": "fa:16:3e:ec:97:fb", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfab305ad-66", "ovs_interfaceid": "fab305ad-6649-4e13-944b-82a2dbc76547", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converting VIF {"id": "fab305ad-6649-4e13-944b-82a2dbc76547", "address": "fa:16:3e:ec:97:fb", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfab305ad-66", "ovs_interfaceid": "fab305ad-6649-4e13-944b-82a2dbc76547", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ec:97:fb,bridge_name='br-int',has_traffic_filtering=True,id=fab305ad-6649-4e13-944b-82a2dbc76547,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfab305ad-66') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.objects.instance [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'pci_devices' on Instance uuid e19eb131-ae90-4220-9d47-15e8b8b332de {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[90252053-ab0d-4038-a7ac-9f042be13542]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:26:59 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:26:59 user nova-compute[71317]:   <uuid>e19eb131-ae90-4220-9d47-15e8b8b332de</uuid>
Apr 14 04:26:59 user nova-compute[71317]:   <name>instance-0000001d</name>
Apr 14 04:26:59 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:26:59 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:26:59 user nova-compute[71317]:   <metadata>
Apr 14 04:26:59 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:26:59 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:26:59 user nova-compute[71317]:       <nova:name>tempest-AttachVolumeTestJSON-server-1852573542</nova:name>
Apr 14 04:26:59 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:26:59</nova:creationTime>
Apr 14 04:26:59 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:26:59 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:26:59 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:26:59 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:26:59 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:26:59 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:26:59 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:26:59 user nova-compute[71317]:       <nova:owner>
Apr 14 04:26:59 user nova-compute[71317]:         <nova:user uuid="7610b45003e34f3395268977612f6c94">tempest-AttachVolumeTestJSON-991906288-project-member</nova:user>
Apr 14 04:26:59 user nova-compute[71317]:         <nova:project uuid="6a89255339f74e55ad7c92967558bb81">tempest-AttachVolumeTestJSON-991906288</nova:project>
Apr 14 04:26:59 user nova-compute[71317]:       </nova:owner>
Apr 14 04:26:59 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:26:59 user nova-compute[71317]:       <nova:ports>
Apr 14 04:26:59 user nova-compute[71317]:         <nova:port uuid="fab305ad-6649-4e13-944b-82a2dbc76547">
Apr 14 04:26:59 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Apr 14 04:26:59 user nova-compute[71317]:         </nova:port>
Apr 14 04:26:59 user nova-compute[71317]:       </nova:ports>
Apr 14 04:26:59 user nova-compute[71317]:     </nova:instance>
Apr 14 04:26:59 user nova-compute[71317]:   </metadata>
Apr 14 04:26:59 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:26:59 user nova-compute[71317]:     <system>
Apr 14 04:26:59 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:26:59 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:26:59 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:26:59 user nova-compute[71317]:       <entry name="serial">e19eb131-ae90-4220-9d47-15e8b8b332de</entry>
Apr 14 04:26:59 user nova-compute[71317]:       <entry name="uuid">e19eb131-ae90-4220-9d47-15e8b8b332de</entry>
Apr 14 04:26:59 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:26:59 user nova-compute[71317]:     </system>
Apr 14 04:26:59 user nova-compute[71317]:   </sysinfo>
Apr 14 04:26:59 user nova-compute[71317]:   <os>
Apr 14 04:26:59 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:26:59 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:26:59 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:26:59 user nova-compute[71317]:   </os>
Apr 14 04:26:59 user nova-compute[71317]:   <features>
Apr 14 04:26:59 user nova-compute[71317]:     <acpi/>
Apr 14 04:26:59 user nova-compute[71317]:     <apic/>
Apr 14 04:26:59 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:26:59 user nova-compute[71317]:   </features>
Apr 14 04:26:59 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:26:59 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:26:59 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:26:59 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:26:59 user nova-compute[71317]:   </clock>
Apr 14 04:26:59 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:26:59 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:26:59 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:26:59 user nova-compute[71317]:   </cpu>
Apr 14 04:26:59 user nova-compute[71317]:   <devices>
Apr 14 04:26:59 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:26:59 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:26:59 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk"/>
Apr 14 04:26:59 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:26:59 user nova-compute[71317]:     </disk>
Apr 14 04:26:59 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:26:59 user nova-compute[71317]:       <mac address="fa:16:3e:ec:97:fb"/>
Apr 14 04:26:59 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:26:59 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:26:59 user nova-compute[71317]:       <target dev="tapfab305ad-66"/>
Apr 14 04:26:59 user nova-compute[71317]:     </interface>
Apr 14 04:26:59 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:26:59 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/console.log" append="off"/>
Apr 14 04:26:59 user nova-compute[71317]:     </serial>
Apr 14 04:26:59 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:26:59 user nova-compute[71317]:     <video>
Apr 14 04:26:59 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:26:59 user nova-compute[71317]:     </video>
Apr 14 04:26:59 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:26:59 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:26:59 user nova-compute[71317]:     </rng>
Apr 14 04:26:59 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:26:59 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:26:59 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:26:59 user nova-compute[71317]:     </memballoon>
Apr 14 04:26:59 user nova-compute[71317]:   </devices>
Apr 14 04:26:59 user nova-compute[71317]: </domain>
Apr 14 04:26:59 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:26:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1852573542',display_name='tempest-AttachVolumeTestJSON-server-1852573542',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1852573542',id=29,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPahbjwxSvxq0IxMz5/M3Sbn33mEC8YBDkFVpCOH/WAboRbjSbxsllitzmJB7SVS4TKY3MCm6q57KW3T5WFQQOYggXTJylYDmiHUt2/y+RsASQ2RPX3ud6TI2ErXR0uk4g==',key_name='tempest-keypair-1690869240',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6a89255339f74e55ad7c92967558bb81',ramdisk_id='',reservation_id='r-j2gf6jl3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-991906288',owner_user_name='tempest-AttachVolumeTestJSON-991906288-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:26:57Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7610b45003e34f3395268977612f6c94',uuid=e19eb131-ae90-4220-9d47-15e8b8b332de,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fab305ad-6649-4e13-944b-82a2dbc76547", "address": "fa:16:3e:ec:97:fb", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfab305ad-66", "ovs_interfaceid": "fab305ad-6649-4e13-944b-82a2dbc76547", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converting VIF {"id": "fab305ad-6649-4e13-944b-82a2dbc76547", "address": "fa:16:3e:ec:97:fb", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfab305ad-66", "ovs_interfaceid": "fab305ad-6649-4e13-944b-82a2dbc76547", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ec:97:fb,bridge_name='br-int',has_traffic_filtering=True,id=fab305ad-6649-4e13-944b-82a2dbc76547,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfab305ad-66') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_vif [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ec:97:fb,bridge_name='br-int',has_traffic_filtering=True,id=fab305ad-6649-4e13-944b-82a2dbc76547,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfab305ad-66') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapfab305ad-66, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapfab305ad-66, col_values=(('external_ids', {'iface-id': 'fab305ad-6649-4e13-944b-82a2dbc76547', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ec:97:fb', 'vm-uuid': 'e19eb131-ae90-4220-9d47-15e8b8b332de'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7e27a47-285e-4e90-92ff-fb7261e029d1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d] added
Apr 14 04:26:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:26:59 user nova-compute[71317]: INFO os_vif [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ec:97:fb,bridge_name='br-int',has_traffic_filtering=True,id=fab305ad-6649-4e13-944b-82a2dbc76547,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfab305ad-66')
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9abdfff8-fc2d-4547-a0ff-fb8fb132e02c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d2d17e2-f79f-43a3-92e7-e9eac1f70b4d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v xaQKgXWG {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No VIF found with MAC fa:16:3e:ec:97:fb, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v xaQKgXWG" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[46eafe0b-a91a-4795-b9b7-a9c21de921bf]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'xaQKgXWG'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[38dfbe9a-02cd-4e9b-abd3-b7f9da04a123]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[de3412ce-a80b-4435-830f-f6d4df931c8c]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:26:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:26:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --login" returned: 0 in 0.083s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[72f20727-bd3a-4fd8-8235-24971ed98ed9]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d, portal: 172.16.0.220,3260]
Apr 14 04:26:59 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d, portal: 172.16.0.220,3260] successful.
Apr 14 04:26:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd3029cc-bf8e-49c8-80b0-5e4d76bdef0b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[555cf2d9-fd4a-4566-b123-73775a4f5e35]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d (non-flash)
Apr 14 04:26:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:26:59 user nova-compute[71317]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d (non-flash)
Apr 14 04:26:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.356s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] HCTL ('38', '-', '-', 0) found on session 32 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[836312c7-b24e-4d72-8410-5c44d813ed99]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Searching for a device in session 32 and hctl ['38', '*', '*', 0] yield: None {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-456ecbd1-5e8c-4aa5-8e23-62f208e174f4 req-ecff8947-5518-41fe-9392-cbcab6490f58 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Updated VIF entry in instance network info cache for port fab305ad-6649-4e13-944b-82a2dbc76547. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-456ecbd1-5e8c-4aa5-8e23-62f208e174f4 req-ecff8947-5518-41fe-9392-cbcab6490f58 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Updating instance_info_cache with network_info: [{"id": "fab305ad-6649-4e13-944b-82a2dbc76547", "address": "fa:16:3e:ec:97:fb", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfab305ad-66", "ovs_interfaceid": "fab305ad-6649-4e13-944b-82a2dbc76547", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:26:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-456ecbd1-5e8c-4aa5-8e23-62f208e174f4 req-ecff8947-5518-41fe-9392-cbcab6490f58 service nova] Releasing lock "refresh_cache-e19eb131-ae90-4220-9d47-15e8b8b332de" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:27:00 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Searching for a device in session 32 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:27:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Connected to sdg using {'target_iqn': 'iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'xaQKgXWG', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:27:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.385s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:27:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] <== connect_volume: return (1386ms) {'type': 'block', 'scsi_wwn': '23139663031623238', 'path': '/dev/sdg'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:27:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23139663031623238', 'path': '/dev/sdg'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:27:00 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'flavor' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:27:00 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:27:00 user nova-compute[71317]:   <source dev="/dev/sdg"/>
Apr 14 04:27:00 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:27:00 user nova-compute[71317]:   <serial>430bf7a7-6067-4516-9a60-f895bded451d</serial>
Apr 14 04:27:00 user nova-compute[71317]: </disk>
Apr 14 04:27:00 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:27:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-c06f3d60-9ef0-46e3-9bd1-5c349e8a59f4 req-0c5e1f8c-ca46-4dae-ab7a-57ac86a6d240 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Received event network-vif-plugged-fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c06f3d60-9ef0-46e3-9bd1-5c349e8a59f4 req-0c5e1f8c-ca46-4dae-ab7a-57ac86a6d240 service nova] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c06f3d60-9ef0-46e3-9bd1-5c349e8a59f4 req-0c5e1f8c-ca46-4dae-ab7a-57ac86a6d240 service nova] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c06f3d60-9ef0-46e3-9bd1-5c349e8a59f4 req-0c5e1f8c-ca46-4dae-ab7a-57ac86a6d240 service nova] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-c06f3d60-9ef0-46e3-9bd1-5c349e8a59f4 req-0c5e1f8c-ca46-4dae-ab7a-57ac86a6d240 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] No waiting events found dispatching network-vif-plugged-fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:01 user nova-compute[71317]: WARNING nova.compute.manager [req-c06f3d60-9ef0-46e3-9bd1-5c349e8a59f4 req-0c5e1f8c-ca46-4dae-ab7a-57ac86a6d240 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Received unexpected event network-vif-plugged-fab305ad-6649-4e13-944b-82a2dbc76547 for instance with vm_state building and task_state spawning.
Apr 14 04:27:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:27:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:27:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No VIF found with MAC fa:16:3e:17:59:61, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:27:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c9e54616-7ddf-4509-ae7c-9cb334e34657 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.464s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:02 user nova-compute[71317]: INFO nova.compute.manager [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Detaching volume 430bf7a7-6067-4516-9a60-f895bded451d
Apr 14 04:27:02 user nova-compute[71317]: INFO nova.virt.block_device [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attempting to driver detach volume 430bf7a7-6067-4516-9a60-f895bded451d from mountpoint /dev/vdb
Apr 14 04:27:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Attempting to detach device vdb from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:27:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:27:02 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:27:02 user nova-compute[71317]:   <source dev="/dev/sdg"/>
Apr 14 04:27:02 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:27:02 user nova-compute[71317]:   <serial>430bf7a7-6067-4516-9a60-f895bded451d</serial>
Apr 14 04:27:02 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:27:02 user nova-compute[71317]: </disk>
Apr 14 04:27:02 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:27:02 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Successfully detached device vdb from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the persistent domain config.
Apr 14 04:27:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:27:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:27:02 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:27:02 user nova-compute[71317]:   <source dev="/dev/sdg"/>
Apr 14 04:27:02 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:27:02 user nova-compute[71317]:   <serial>430bf7a7-6067-4516-9a60-f895bded451d</serial>
Apr 14 04:27:02 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:27:02 user nova-compute[71317]: </disk>
Apr 14 04:27:02 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446423.0246205, f593ceb0-de44-490b-b365-3a05a3acd066 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:27:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Successfully detached device vdb from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the live domain config.
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'xaQKgXWG', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:27:03 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[be76b019-9154-4dd2-941b-0ef703a34349]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:27:03 user nova-compute[71317]: No targets found.
Apr 14 04:27:03 user nova-compute[71317]: iSNS:
Apr 14 04:27:03 user nova-compute[71317]: No targets found.
Apr 14 04:27:03 user nova-compute[71317]: STATIC:
Apr 14 04:27:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f
Apr 14 04:27:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d
Apr 14 04:27:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee
Apr 14 04:27:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:27:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4
Apr 14 04:27:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:27:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:03 user nova-compute[71317]: FIRMWARE:
Apr 14 04:27:03 user nova-compute[71317]: No targets found.
Apr 14 04:27:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:27:03 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:27:03 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a4c61406-3b36-4f64-9f8f-079312a0c4a4]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3250869-efa5-4bc2-a9b1-198e8711496b]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:03 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:03 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:03 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:03 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:03 user nova-compute[71317]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d (non-flash)
Apr 14 04:27:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:03 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:03 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:03 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:03 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:03 user nova-compute[71317]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d (non-flash)
Apr 14 04:27:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c43b45e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d'): ({'sdg'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Removing single pathed devices sdg {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[be524124-e66b-4bc5-87f8-c2df774c1cab]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.compute.manager [req-b4902f2c-3bb0-4fbb-accb-76bbedc5dbb7 req-04d439b5-024d-4321-996a-9a98b7e044bc service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Received event network-vif-plugged-fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b4902f2c-3bb0-4fbb-accb-76bbedc5dbb7 req-04d439b5-024d-4321-996a-9a98b7e044bc service nova] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b4902f2c-3bb0-4fbb-accb-76bbedc5dbb7 req-04d439b5-024d-4321-996a-9a98b7e044bc service nova] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b4902f2c-3bb0-4fbb-accb-76bbedc5dbb7 req-04d439b5-024d-4321-996a-9a98b7e044bc service nova] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.compute.manager [req-b4902f2c-3bb0-4fbb-accb-76bbedc5dbb7 req-04d439b5-024d-4321-996a-9a98b7e044bc service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] No waiting events found dispatching network-vif-plugged-fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:03 user nova-compute[71317]: WARNING nova.compute.manager [req-b4902f2c-3bb0-4fbb-accb-76bbedc5dbb7 req-04d439b5-024d-4321-996a-9a98b7e044bc service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Received unexpected event network-vif-plugged-fab305ad-6649-4e13-944b-82a2dbc76547 for instance with vm_state building and task_state spawning.
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e4deba8-44e1-4df6-9004-733bcce982f0]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Flushing IO for device /dev/sdg {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f529a29-e9f1-43d0-a7bf-a2a210ecbbdc]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.044s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a4c92902-659f-4304-af8f-aba3a5e957bc]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] SCSI volumes sdg have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a00ba967-7b29-4256-93cb-8d5c593c791b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --logout" returned: 0 in 0.066s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fda19781-e96b-4ea3-b7e3-a3209ef5a56e]: (4, ('Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d, portal: 172.16.0.220,3260]\nLogout of [sid: 32, target: iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d, portal: 172.16.0.220,3260]
Apr 14 04:27:03 user nova-compute[71317]: Logout of [sid: 32, target: iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d, portal: 172.16.0.220,3260] successful.
Apr 14 04:27:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:430bf7a7-6067-4516-9a60-f895bded451d -p 172.16.0.220:3260 --op delete" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d765ecc-6e0b-4ee5-8572-6282a6569c82]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.322s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] <== disconnect_volume: return (324ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446423.3784175, e19eb131-ae90-4220-9d47-15e8b8b332de => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:27:03 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] VM Resumed (Lifecycle Event)
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:27:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Instance spawned successfully.
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:27:03 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446423.3801212, e19eb131-ae90-4220-9d47-15e8b8b332de => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:27:03 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] VM Started (Lifecycle Event)
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:27:03 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:27:03 user nova-compute[71317]: INFO nova.compute.manager [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Took 5.98 seconds to spawn the instance on the hypervisor.
Apr 14 04:27:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:27:03 user nova-compute[71317]: INFO nova.compute.manager [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Took 6.78 seconds to build instance.
Apr 14 04:27:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-054e212a-d99b-48dc-b578-b949add9b0f3 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.881s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:04 user nova-compute[71317]: DEBUG nova.objects.instance [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'flavor' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-0cb06385-90ad-41ad-b17a-e3dee2a96e6c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.681s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-ed93efce-db17-46d5-b1bb-ee2767e07b06 req-fdc5e096-caff-4891-b2a7-0e49089ff91f service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-changed-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-ed93efce-db17-46d5-b1bb-ee2767e07b06 req-fdc5e096-caff-4891-b2a7-0e49089ff91f service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Refreshing instance network info cache due to event network-changed-e3868733-d538-4469-9a3e-c02014ec9066. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ed93efce-db17-46d5-b1bb-ee2767e07b06 req-fdc5e096-caff-4891-b2a7-0e49089ff91f service nova] Acquiring lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ed93efce-db17-46d5-b1bb-ee2767e07b06 req-fdc5e096-caff-4891-b2a7-0e49089ff91f service nova] Acquired lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG nova.network.neutron [req-ed93efce-db17-46d5-b1bb-ee2767e07b06 req-fdc5e096-caff-4891-b2a7-0e49089ff91f service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Refreshing network info cache for port e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG nova.objects.instance [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'flavor' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.052s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:07 user nova-compute[71317]: INFO nova.compute.manager [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Attaching volume 2ea04249-09da-471f-a8b5-89a901381329 to /dev/vdb
Apr 14 04:27:07 user nova-compute[71317]: DEBUG os_brick.utils [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a6f1cb8-8468-474d-92de-084b6c4b07c8]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3455e90e-d4ae-4b94-9701-1944dce75261]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3c37f8c-88ef-4e3b-bad3-8db5a68389dd]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[54a1b837-ee75-4e3f-b8e3-b8a5b96f3a7d]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:27:07 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:27:07 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2175c113-d215-4f0a-bc91-b7489fb7481e]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG os_brick.utils [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] <== get_connector_properties: return (144ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:27:07 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updating existing volume attachment record: e2a40397-a048-409f-b16d-fcaf23668422 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:27:08 user nova-compute[71317]: DEBUG nova.network.neutron [req-ed93efce-db17-46d5-b1bb-ee2767e07b06 req-fdc5e096-caff-4891-b2a7-0e49089ff91f service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updated VIF entry in instance network info cache for port e3868733-d538-4469-9a3e-c02014ec9066. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:27:08 user nova-compute[71317]: DEBUG nova.network.neutron [req-ed93efce-db17-46d5-b1bb-ee2767e07b06 req-fdc5e096-caff-4891-b2a7-0e49089ff91f service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updating instance_info_cache with network_info: [{"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:27:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ed93efce-db17-46d5-b1bb-ee2767e07b06 req-fdc5e096-caff-4891-b2a7-0e49089ff91f service nova] Releasing lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:27:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dWfGZPuJ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:27:12 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:27:12 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:27:12 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260" returned: 21 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2e04908-925b-493f-a060-d24b88103eb9]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:27:12 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7fe2220d-bb24-4792-ade3-2d58b26165f0]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329] added
Apr 14 04:27:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f0e94a9-578b-48b3-8867-63879da64625]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[79e27be8-309f-46aa-b3a8-865c6872e79e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v dWfGZPuJ {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v dWfGZPuJ" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3fc414c0-de90-49a1-a26c-a17aeca15f15]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'dWfGZPuJ'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3bdac21-522b-4d27-8719-415cbe3fc34f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[11983cee-e603-461b-a66e-c992c41c8751]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --login" returned: 0 in 0.075s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[afb5d374-6630-455a-85c5-a6ad76f00c18]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329, portal: 172.16.0.220,3260]
Apr 14 04:27:12 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329, portal: 172.16.0.220,3260] successful.
Apr 14 04:27:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b91833ce-aa9c-48c2-830d-abafae82e08a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[86a2f4a6-8b33-46f6-b1c8-386c86ca8f43]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.307s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] HCTL ('38', '-', '-', 0) found on session 33 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b6b50d6-ab06-448e-83a5-a59f5635bb68]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Searching for a device in session 33 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:27:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Connected to sdg using {'target_iqn': 'iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dWfGZPuJ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:27:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.344s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:27:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] <== connect_volume: return (1346ms) {'type': 'block', 'scsi_wwn': '26438633635343165', 'path': '/dev/sdg'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:27:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26438633635343165', 'path': '/dev/sdg'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:27:13 user nova-compute[71317]: DEBUG nova.objects.instance [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'flavor' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:27:13 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:27:13 user nova-compute[71317]:   <source dev="/dev/sdg"/>
Apr 14 04:27:13 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:27:13 user nova-compute[71317]:   <serial>2ea04249-09da-471f-a8b5-89a901381329</serial>
Apr 14 04:27:13 user nova-compute[71317]: </disk>
Apr 14 04:27:13 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:27:14 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:27:14 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:27:14 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No VIF found with MAC fa:16:3e:71:08:35, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:27:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fe9b03b-4129-4eef-b04c-80c3b2c2300b tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.672s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:15 user nova-compute[71317]: INFO nova.compute.manager [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Rescuing
Apr 14 04:27:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:27:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquired lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:27:15 user nova-compute[71317]: DEBUG nova.network.neutron [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:27:16 user nova-compute[71317]: DEBUG nova.network.neutron [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updating instance_info_cache with network_info: [{"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:27:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Releasing lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:27:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:16 user nova-compute[71317]: DEBUG nova.compute.manager [req-093bf229-6fec-425c-a49e-41f2dd2fa49a req-8e1415c5-5e99-4753-a85d-86821c4c0c97 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-unplugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-093bf229-6fec-425c-a49e-41f2dd2fa49a req-8e1415c5-5e99-4753-a85d-86821c4c0c97 service nova] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-093bf229-6fec-425c-a49e-41f2dd2fa49a req-8e1415c5-5e99-4753-a85d-86821c4c0c97 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-093bf229-6fec-425c-a49e-41f2dd2fa49a req-8e1415c5-5e99-4753-a85d-86821c4c0c97 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:16 user nova-compute[71317]: DEBUG nova.compute.manager [req-093bf229-6fec-425c-a49e-41f2dd2fa49a req-8e1415c5-5e99-4753-a85d-86821c4c0c97 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] No waiting events found dispatching network-vif-unplugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:16 user nova-compute[71317]: WARNING nova.compute.manager [req-093bf229-6fec-425c-a49e-41f2dd2fa49a req-8e1415c5-5e99-4753-a85d-86821c4c0c97 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received unexpected event network-vif-unplugged-e3868733-d538-4469-9a3e-c02014ec9066 for instance with vm_state active and task_state rescuing.
Apr 14 04:27:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Instance destroyed successfully.
Apr 14 04:27:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Attempting rescue
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=71317) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Instance directory exists: not creating {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 04:27:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Creating image(s)
Apr 14 04:27:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "/opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "/opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "/opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'trusted_certs' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.rescue {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.rescue" returned: 0 in 0.065s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.205s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'migration_context' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Start _get_guest_xml network_info=[{"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "vif_mac": "fa:16:3e:71:08:35"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'resources' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'numa_topology' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:17 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:27:17 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'vcpu_model' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:25:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1081038034',display_name='tempest-ServerRescueNegativeTestJSON-server-1081038034',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1081038034',id=25,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCb+23dydPc8XvPesmHXUsmKMTGTpSa5GCZHGQJYeeij6pIqTWqaFg9yzcA/VhrLxaLND3aQCfGfHGfvo53ucWJsjWgHFiCEptESTMFy12g7udsn7vnD6U5hRBURzXxHFg==',key_name='tempest-keypair-1320453961',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:25:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='33320dc2f2c14ecfa5abf6574029c36c',ramdisk_id='',reservation_id='r-ym9z46z3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-508041341',owner_user_name='tempest-ServerRescueNegativeTestJSON-508041341-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:25:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5234914aec8e443b8a38ac6b38729031',uuid=cab84e5c-99ba-4e61-bf68-0a02b31ded5c,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "vif_mac": "fa:16:3e:71:08:35"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converting VIF {"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "vif_mac": "fa:16:3e:71:08:35"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:71:08:35,bridge_name='br-int',has_traffic_filtering=True,id=e3868733-d538-4469-9a3e-c02014ec9066,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3868733-d5') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'pci_devices' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:27:17 user nova-compute[71317]:   <uuid>cab84e5c-99ba-4e61-bf68-0a02b31ded5c</uuid>
Apr 14 04:27:17 user nova-compute[71317]:   <name>instance-00000019</name>
Apr 14 04:27:17 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:27:17 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:27:17 user nova-compute[71317]:   <metadata>
Apr 14 04:27:17 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:27:17 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:27:17 user nova-compute[71317]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-1081038034</nova:name>
Apr 14 04:27:17 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:27:17</nova:creationTime>
Apr 14 04:27:17 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:27:17 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:27:17 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:27:17 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:27:17 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:27:17 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:27:17 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:27:17 user nova-compute[71317]:       <nova:owner>
Apr 14 04:27:17 user nova-compute[71317]:         <nova:user uuid="5234914aec8e443b8a38ac6b38729031">tempest-ServerRescueNegativeTestJSON-508041341-project-member</nova:user>
Apr 14 04:27:17 user nova-compute[71317]:         <nova:project uuid="33320dc2f2c14ecfa5abf6574029c36c">tempest-ServerRescueNegativeTestJSON-508041341</nova:project>
Apr 14 04:27:17 user nova-compute[71317]:       </nova:owner>
Apr 14 04:27:17 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:27:17 user nova-compute[71317]:       <nova:ports>
Apr 14 04:27:17 user nova-compute[71317]:         <nova:port uuid="e3868733-d538-4469-9a3e-c02014ec9066">
Apr 14 04:27:17 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 04:27:17 user nova-compute[71317]:         </nova:port>
Apr 14 04:27:17 user nova-compute[71317]:       </nova:ports>
Apr 14 04:27:17 user nova-compute[71317]:     </nova:instance>
Apr 14 04:27:17 user nova-compute[71317]:   </metadata>
Apr 14 04:27:17 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:27:17 user nova-compute[71317]:     <system>
Apr 14 04:27:17 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:27:17 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:27:17 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:27:17 user nova-compute[71317]:       <entry name="serial">cab84e5c-99ba-4e61-bf68-0a02b31ded5c</entry>
Apr 14 04:27:17 user nova-compute[71317]:       <entry name="uuid">cab84e5c-99ba-4e61-bf68-0a02b31ded5c</entry>
Apr 14 04:27:17 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:27:17 user nova-compute[71317]:     </system>
Apr 14 04:27:17 user nova-compute[71317]:   </sysinfo>
Apr 14 04:27:17 user nova-compute[71317]:   <os>
Apr 14 04:27:17 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:27:17 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:27:17 user nova-compute[71317]:   </os>
Apr 14 04:27:17 user nova-compute[71317]:   <features>
Apr 14 04:27:17 user nova-compute[71317]:     <acpi/>
Apr 14 04:27:17 user nova-compute[71317]:     <apic/>
Apr 14 04:27:17 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:27:17 user nova-compute[71317]:   </features>
Apr 14 04:27:17 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:27:17 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:27:17 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:27:17 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:27:17 user nova-compute[71317]:   </clock>
Apr 14 04:27:17 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:27:17 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:27:17 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:27:17 user nova-compute[71317]:   </cpu>
Apr 14 04:27:17 user nova-compute[71317]:   <devices>
Apr 14 04:27:17 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:27:17 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:27:17 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.rescue"/>
Apr 14 04:27:17 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:27:17 user nova-compute[71317]:     </disk>
Apr 14 04:27:17 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:27:17 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:27:17 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk"/>
Apr 14 04:27:17 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:27:17 user nova-compute[71317]:     </disk>
Apr 14 04:27:17 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:27:17 user nova-compute[71317]:       <mac address="fa:16:3e:71:08:35"/>
Apr 14 04:27:17 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:27:17 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:27:17 user nova-compute[71317]:       <target dev="tape3868733-d5"/>
Apr 14 04:27:17 user nova-compute[71317]:     </interface>
Apr 14 04:27:17 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:27:17 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/console.log" append="off"/>
Apr 14 04:27:17 user nova-compute[71317]:     </serial>
Apr 14 04:27:17 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:27:17 user nova-compute[71317]:     <video>
Apr 14 04:27:17 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:27:17 user nova-compute[71317]:     </video>
Apr 14 04:27:17 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:27:17 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:27:17 user nova-compute[71317]:     </rng>
Apr 14 04:27:17 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:27:17 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:27:17 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:27:17 user nova-compute[71317]:     </memballoon>
Apr 14 04:27:17 user nova-compute[71317]:   </devices>
Apr 14 04:27:17 user nova-compute[71317]: </domain>
Apr 14 04:27:17 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:27:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Instance destroyed successfully.
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] No VIF found with MAC fa:16:3e:71:08:35, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:27:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-f3d3e995-8666-4d8a-a2b4-4303b90cd7ed req-4b44ad71-6848-4b74-ac4a-771b5faac4df service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f3d3e995-8666-4d8a-a2b4-4303b90cd7ed req-4b44ad71-6848-4b74-ac4a-771b5faac4df service nova] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f3d3e995-8666-4d8a-a2b4-4303b90cd7ed req-4b44ad71-6848-4b74-ac4a-771b5faac4df service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f3d3e995-8666-4d8a-a2b4-4303b90cd7ed req-4b44ad71-6848-4b74-ac4a-771b5faac4df service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-f3d3e995-8666-4d8a-a2b4-4303b90cd7ed req-4b44ad71-6848-4b74-ac4a-771b5faac4df service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] No waiting events found dispatching network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:18 user nova-compute[71317]: WARNING nova.compute.manager [req-f3d3e995-8666-4d8a-a2b4-4303b90cd7ed req-4b44ad71-6848-4b74-ac4a-771b5faac4df service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received unexpected event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 for instance with vm_state active and task_state rescuing.
Apr 14 04:27:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-ef392c46-9008-415f-9d40-91829e7299b8 req-ab4d01cd-94ae-42cb-a51a-27e0de20fc4f service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ef392c46-9008-415f-9d40-91829e7299b8 req-ab4d01cd-94ae-42cb-a51a-27e0de20fc4f service nova] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ef392c46-9008-415f-9d40-91829e7299b8 req-ab4d01cd-94ae-42cb-a51a-27e0de20fc4f service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ef392c46-9008-415f-9d40-91829e7299b8 req-ab4d01cd-94ae-42cb-a51a-27e0de20fc4f service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-ef392c46-9008-415f-9d40-91829e7299b8 req-ab4d01cd-94ae-42cb-a51a-27e0de20fc4f service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] No waiting events found dispatching network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:20 user nova-compute[71317]: WARNING nova.compute.manager [req-ef392c46-9008-415f-9d40-91829e7299b8 req-ab4d01cd-94ae-42cb-a51a-27e0de20fc4f service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received unexpected event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 for instance with vm_state active and task_state rescuing.
Apr 14 04:27:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-ef392c46-9008-415f-9d40-91829e7299b8 req-ab4d01cd-94ae-42cb-a51a-27e0de20fc4f service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ef392c46-9008-415f-9d40-91829e7299b8 req-ab4d01cd-94ae-42cb-a51a-27e0de20fc4f service nova] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ef392c46-9008-415f-9d40-91829e7299b8 req-ab4d01cd-94ae-42cb-a51a-27e0de20fc4f service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ef392c46-9008-415f-9d40-91829e7299b8 req-ab4d01cd-94ae-42cb-a51a-27e0de20fc4f service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-ef392c46-9008-415f-9d40-91829e7299b8 req-ab4d01cd-94ae-42cb-a51a-27e0de20fc4f service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] No waiting events found dispatching network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:20 user nova-compute[71317]: WARNING nova.compute.manager [req-ef392c46-9008-415f-9d40-91829e7299b8 req-ab4d01cd-94ae-42cb-a51a-27e0de20fc4f service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received unexpected event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 for instance with vm_state active and task_state rescuing.
Apr 14 04:27:20 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for cab84e5c-99ba-4e61-bf68-0a02b31ded5c due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:27:20 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446440.9920652, cab84e5c-99ba-4e61-bf68-0a02b31ded5c => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:27:20 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] VM Resumed (Lifecycle Event)
Apr 14 04:27:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-91e87e29-15fc-4275-8cf3-e205b74f56de tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:27:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:27:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:27:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 14 04:27:21 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446440.9932659, cab84e5c-99ba-4e61-bf68-0a02b31ded5c => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:27:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] VM Started (Lifecycle Event)
Apr 14 04:27:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:27:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:27:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:26 user nova-compute[71317]: DEBUG nova.network.neutron [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Port 3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 binding to destination host user is already ACTIVE {{(pid=71317) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3161}}
Apr 14 04:27:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:27:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquired lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:27:26 user nova-compute[71317]: DEBUG nova.network.neutron [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG nova.network.neutron [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating instance_info_cache with network_info: [{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Releasing lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Creating tmpfile /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/tmprd8xr02r to verify with other compute node that the instance is on the same shared storage. {{(pid=71317) check_instance_shared_storage_local /opt/stack/nova/nova/virt/libvirt/driver.py:9595}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-97c65b20-f26d-4b9a-a71f-785198e256b3 req-eae516cc-cff7-4102-b895-6574e2686c83 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-unplugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-97c65b20-f26d-4b9a-a71f-785198e256b3 req-eae516cc-cff7-4102-b895-6574e2686c83 service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-97c65b20-f26d-4b9a-a71f-785198e256b3 req-eae516cc-cff7-4102-b895-6574e2686c83 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-97c65b20-f26d-4b9a-a71f-785198e256b3 req-eae516cc-cff7-4102-b895-6574e2686c83 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-97c65b20-f26d-4b9a-a71f-785198e256b3 req-eae516cc-cff7-4102-b895-6574e2686c83 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-unplugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:27 user nova-compute[71317]: WARNING nova.compute.manager [req-97c65b20-f26d-4b9a-a71f-785198e256b3 req-eae516cc-cff7-4102-b895-6574e2686c83 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-unplugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state resized and task_state resize_reverting.
Apr 14 04:27:27 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance destroyed successfully.
Apr 14 04:27:27 user nova-compute[71317]: DEBUG nova.objects.instance [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'resources' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1350614351',display_name='tempest-ServerActionsTestJSON-server-1398117926',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1350614351',id=6,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:25:54Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={rebuild='server'},migration_context=<?>,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-789bnjb3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:25:54Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=a0a20aeb-ed06-4c65-b87d-74f8f825457d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG os_vif [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3a3e3b13-7b, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:27:27 user nova-compute[71317]: INFO os_vif [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b')
Apr 14 04:27:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:27:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MjAZDWGc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:27:27 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:27:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5782614d-2831-4faf-a74a-a51aa69cdbb0]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:27:28 user nova-compute[71317]: No targets found.
Apr 14 04:27:28 user nova-compute[71317]: iSNS:
Apr 14 04:27:28 user nova-compute[71317]: No targets found.
Apr 14 04:27:28 user nova-compute[71317]: STATIC:
Apr 14 04:27:28 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:27:28 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:28 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:28 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f
Apr 14 04:27:28 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:28 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:28 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee
Apr 14 04:27:28 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:28 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:28 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:27:28 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:28 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:28 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4
Apr 14 04:27:28 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:28 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:28 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:27:28 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:28 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:28 user nova-compute[71317]: FIRMWARE:
Apr 14 04:27:28 user nova-compute[71317]: No targets found.
Apr 14 04:27:28 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:27:28 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:27:28 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[df61b075-aea1-4792-a874-fe31336bfde7]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[74bb38a4-03a1-45ef-9095-2d3558adfc3b]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:28 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:28 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:28 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:28 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:28 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:28 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:28 user nova-compute[71317]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:28 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:28 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:28 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:28 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:28 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f849076af80>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2'): ({'sdd'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Removing single pathed devices sdd {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f623891a-4ec2-4c21-b66b-133090174365]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e21a98d-5662-4d9f-a544-a8eea79540ef]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flushing IO for device /dev/sdd {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc67e257-3f1f-43d6-8a33-05ad8a10cf49]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.043s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[69be4e63-9cd3-44a9-87ea-3d2ca1d02c47]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f264ff08-fa05-44e6-b942-2c8458659487]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --logout" returned: 0 in 0.073s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5da1dd33-cd8e-480a-ba2a-10ca1272f0be]: (4, ('Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260]\nLogout of [sid: 27, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260]
Apr 14 04:27:28 user nova-compute[71317]: Logout of [sid: 27, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260] successful.
Apr 14 04:27:28 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[821920fd-7095-4cfc-87f2-ed098cd305f4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.356s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== disconnect_volume: return (358ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:27:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG nova.objects.instance [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'migration_context' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG nova.compute.manager [req-584d0eec-608f-4d31-aed1-7daec5ab3149 req-f4cbd38c-9db7-4a8c-bf57-ba273518e55a service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-584d0eec-608f-4d31-aed1-7daec5ab3149 req-f4cbd38c-9db7-4a8c-bf57-ba273518e55a service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-584d0eec-608f-4d31-aed1-7daec5ab3149 req-f4cbd38c-9db7-4a8c-bf57-ba273518e55a service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-584d0eec-608f-4d31-aed1-7daec5ab3149 req-f4cbd38c-9db7-4a8c-bf57-ba273518e55a service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG nova.compute.manager [req-584d0eec-608f-4d31-aed1-7daec5ab3149 req-f4cbd38c-9db7-4a8c-bf57-ba273518e55a service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:29 user nova-compute[71317]: WARNING nova.compute.manager [req-584d0eec-608f-4d31-aed1-7daec5ab3149 req-f4cbd38c-9db7-4a8c-bf57-ba273518e55a service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state resized and task_state resize_reverting.
Apr 14 04:27:29 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating instance_info_cache with network_info: [{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:27:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" :: held 0.529s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:30 user nova-compute[71317]: INFO nova.compute.manager [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Swapping old allocation on dict_keys(['e35a99e4-b1ea-4792-a9ff-f93045b2018b']) held by migration c4903c05-a131-4f02-b6b6-805ce86d54cd for instance
Apr 14 04:27:30 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Overwriting current allocation {'allocations': {'e35a99e4-b1ea-4792-a9ff-f93045b2018b': {'resources': {'VCPU': 1, 'MEMORY_MB': 192, 'DISK_GB': 1}, 'generation': 33}}, 'project_id': '8c14c332dfcc44b49ddf3cb482dd0d4c', 'user_id': 'e35b3ae371cf4dd4b21e2999e58cd2f4', 'consumer_generation': 1} on consumer a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) move_allocations /opt/stack/nova/nova/scheduler/client/report.py:2018}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquired lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG nova.network.neutron [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG nova.network.neutron [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating instance_info_cache with network_info: [{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Releasing lock "refresh_cache-a0a20aeb-ed06-4c65-b87d-74f8f825457d" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG os_brick.utils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f44f69d6-5f42-4433-9a51-5bb923f4acfa]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[66221c0c-224b-40a0-a327-7c0c30f39ebd]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[464406c8-889b-4aa7-a408-6e7b4bc89bb6]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2c5a3cb-f154-439a-ae7f-24991889d890]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:27:30 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:27:30 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fc96b12-04cc-4928-8cd5-f9477dd357e9]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:27:30 user nova-compute[71317]: DEBUG os_brick.utils [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== get_connector_properties: return (127ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:27:31 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:27:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:27:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 04:27:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] There are 0 instances to clean {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 04:27:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:34 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.152s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Starting finish_revert_migration {{(pid=71317) finish_revert_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11792}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Start _get_guest_xml network_info=[{"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': None, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MjAZDWGc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': 'a0a20aeb-ed06-4c65-b87d-74f8f825457d', 'attached_at': '2023-04-14T04:27:36.000000', 'detached_at': '', 'volume_id': '9cdf2c58-c295-483e-b766-13e01129ede2', 'serial': '9cdf2c58-c295-483e-b766-13e01129ede2'}, 'attachment_id': '62991587-6529-4cf6-9702-0ab7621d58ca', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:27:35 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:27:35 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.objects.instance [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'vcpu_model' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MjAZDWGc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:27:35 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:27:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:27:35 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:27:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:27:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:27:35 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:27:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260" returned: 21 in 0.374s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d013bdc4-fdfa-47ab-ac2e-42a4aaa4667b]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:27:36 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[86395676-ef10-478a-9759-68ec394e32cc]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2] added
Apr 14 04:27:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3cb72bc-2984-40ff-ac75-9aa3393c2ef5]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1975e66-b6ea-41d4-897e-8393ff80d2b7]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v MjAZDWGc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v MjAZDWGc" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[79aaeace-3c08-4ec9-a561-0a50022ede87]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'MjAZDWGc'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a199635-5798-47d7-b07d-b99b141f6469]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json" returned: 0 in 0.205s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c406bb9b-56cf-44fc-9525-2980933763fc]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --login" returned: 0 in 0.076s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e87270e6-656e-45ce-822c-5226f640a783]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260]
Apr 14 04:27:36 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260] successful.
Apr 14 04:27:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f43d463-eefb-4c28-8c83-cd75955fe914]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7712621e-c4f0-4588-9840-769f739919a8]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.808s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] HCTL ('35', '-', '-', 0) found on session 34 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7829bda4-9750-4c7e-8e09-32baa367501b]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Searching for a device in session 34 and hctl ['35', '*', '*', 0] yield: None {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Searching for a device in session 34 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MjAZDWGc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.862s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== connect_volume: return (1864ms) {'type': 'block', 'scsi_wwn': '23661646666666433', 'path': '/dev/sdd'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23661646666666433', 'path': '/dev/sdd'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1350614351',display_name='tempest-ServerActionsTestJSON-server-1398117926',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1350614351',id=6,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:25:54Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-789bnjb3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:27:26Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=a0a20aeb-ed06-4c65-b87d-74f8f825457d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:27:37 user nova-compute[71317]:   <uuid>a0a20aeb-ed06-4c65-b87d-74f8f825457d</uuid>
Apr 14 04:27:37 user nova-compute[71317]:   <name>instance-00000006</name>
Apr 14 04:27:37 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:27:37 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:27:37 user nova-compute[71317]:   <metadata>
Apr 14 04:27:37 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:27:37 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:27:37 user nova-compute[71317]:       <nova:name>tempest-ServerActionsTestJSON-server-1398117926</nova:name>
Apr 14 04:27:37 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:27:35</nova:creationTime>
Apr 14 04:27:37 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:27:37 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:27:37 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:27:37 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:27:37 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:27:37 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:27:37 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:27:37 user nova-compute[71317]:       <nova:owner>
Apr 14 04:27:37 user nova-compute[71317]:         <nova:user uuid="e35b3ae371cf4dd4b21e2999e58cd2f4">tempest-ServerActionsTestJSON-1048786641-project-member</nova:user>
Apr 14 04:27:37 user nova-compute[71317]:         <nova:project uuid="8c14c332dfcc44b49ddf3cb482dd0d4c">tempest-ServerActionsTestJSON-1048786641</nova:project>
Apr 14 04:27:37 user nova-compute[71317]:       </nova:owner>
Apr 14 04:27:37 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:27:37 user nova-compute[71317]:       <nova:ports>
Apr 14 04:27:37 user nova-compute[71317]:         <nova:port uuid="3a3e3b13-7b4e-46c8-9123-5c1f121a38b9">
Apr 14 04:27:37 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 04:27:37 user nova-compute[71317]:         </nova:port>
Apr 14 04:27:37 user nova-compute[71317]:       </nova:ports>
Apr 14 04:27:37 user nova-compute[71317]:     </nova:instance>
Apr 14 04:27:37 user nova-compute[71317]:   </metadata>
Apr 14 04:27:37 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:27:37 user nova-compute[71317]:     <system>
Apr 14 04:27:37 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:27:37 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:27:37 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:27:37 user nova-compute[71317]:       <entry name="serial">a0a20aeb-ed06-4c65-b87d-74f8f825457d</entry>
Apr 14 04:27:37 user nova-compute[71317]:       <entry name="uuid">a0a20aeb-ed06-4c65-b87d-74f8f825457d</entry>
Apr 14 04:27:37 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:27:37 user nova-compute[71317]:     </system>
Apr 14 04:27:37 user nova-compute[71317]:   </sysinfo>
Apr 14 04:27:37 user nova-compute[71317]:   <os>
Apr 14 04:27:37 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:27:37 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:27:37 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:27:37 user nova-compute[71317]:   </os>
Apr 14 04:27:37 user nova-compute[71317]:   <features>
Apr 14 04:27:37 user nova-compute[71317]:     <acpi/>
Apr 14 04:27:37 user nova-compute[71317]:     <apic/>
Apr 14 04:27:37 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:27:37 user nova-compute[71317]:   </features>
Apr 14 04:27:37 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:27:37 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:27:37 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:27:37 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:27:37 user nova-compute[71317]:   </clock>
Apr 14 04:27:37 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:27:37 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:27:37 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:27:37 user nova-compute[71317]:   </cpu>
Apr 14 04:27:37 user nova-compute[71317]:   <devices>
Apr 14 04:27:37 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:27:37 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:27:37 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk"/>
Apr 14 04:27:37 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:27:37 user nova-compute[71317]:     </disk>
Apr 14 04:27:37 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:27:37 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:27:37 user nova-compute[71317]:       <source dev="/dev/sdd"/>
Apr 14 04:27:37 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:27:37 user nova-compute[71317]:       <serial>9cdf2c58-c295-483e-b766-13e01129ede2</serial>
Apr 14 04:27:37 user nova-compute[71317]:     </disk>
Apr 14 04:27:37 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:27:37 user nova-compute[71317]:       <mac address="fa:16:3e:5d:8f:83"/>
Apr 14 04:27:37 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:27:37 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:27:37 user nova-compute[71317]:       <target dev="tap3a3e3b13-7b"/>
Apr 14 04:27:37 user nova-compute[71317]:     </interface>
Apr 14 04:27:37 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:27:37 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/console.log" append="off"/>
Apr 14 04:27:37 user nova-compute[71317]:     </serial>
Apr 14 04:27:37 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:27:37 user nova-compute[71317]:     <video>
Apr 14 04:27:37 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:27:37 user nova-compute[71317]:     </video>
Apr 14 04:27:37 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:27:37 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:27:37 user nova-compute[71317]:     </rng>
Apr 14 04:27:37 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:27:37 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:27:37 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:27:37 user nova-compute[71317]:     </memballoon>
Apr 14 04:27:37 user nova-compute[71317]:   </devices>
Apr 14 04:27:37 user nova-compute[71317]: </domain>
Apr 14 04:27:37 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1350614351',display_name='tempest-ServerActionsTestJSON-server-1398117926',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1350614351',id=6,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:25:54Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-789bnjb3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:27:26Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=a0a20aeb-ed06-4c65-b87d-74f8f825457d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG os_vif [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3a3e3b13-7b, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap3a3e3b13-7b, col_values=(('external_ids', {'iface-id': '3a3e3b13-7b4e-46c8-9123-5c1f121a38b9', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:5d:8f:83', 'vm-uuid': 'a0a20aeb-ed06-4c65-b87d-74f8f825457d'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:27:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:37 user nova-compute[71317]: INFO os_vif [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b')
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.rescue --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.rescue --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.rescue --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75/disk.rescue --force-share --output=json" returned: 0 in 0.152s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-e698e99c-11b7-45f4-8d8d-e339c567fb43 req-d27e0c0a-5fc1-4893-b392-1e53e8b8e104 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e698e99c-11b7-45f4-8d8d-e339c567fb43 req-d27e0c0a-5fc1-4893-b392-1e53e8b8e104 service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e698e99c-11b7-45f4-8d8d-e339c567fb43 req-d27e0c0a-5fc1-4893-b392-1e53e8b8e104 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e698e99c-11b7-45f4-8d8d-e339c567fb43 req-d27e0c0a-5fc1-4893-b392-1e53e8b8e104 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:39 user nova-compute[71317]: DEBUG nova.compute.manager [req-e698e99c-11b7-45f4-8d8d-e339c567fb43 req-d27e0c0a-5fc1-4893-b392-1e53e8b8e104 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:39 user nova-compute[71317]: WARNING nova.compute.manager [req-e698e99c-11b7-45f4-8d8d-e339c567fb43 req-d27e0c0a-5fc1-4893-b392-1e53e8b8e104 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state resized and task_state resize_reverting.
Apr 14 04:27:39 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:27:39 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:27:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7278MB free_disk=26.222999572753906GB free_vcpus=1 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:27:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance f593ceb0-de44-490b-b365-3a05a3acd066 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 5273d202-ffc5-43c6-bb4f-f6d1a6855271 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 4a1c695b-2ca4-4652-8fb6-19accdab6ebe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance e19eb131-ae90-4220-9d47-15e8b8b332de actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 12 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=2048MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=12 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing inventories for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating ProviderTree inventory for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating inventory in ProviderTree for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing aggregate associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, aggregates: None {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing trait associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, traits: COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSE41,HW_CPU_X86_SSE42,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_TRUSTED_CERTS,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_ACCELERATORS,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NET_VIF_MODEL_E1000,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_NODE,HW_CPU_X86_SSE2,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,HW_CPU_X86_SSSE3,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_STORAGE_BUS_USB,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_NE2K_PCI {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:27:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.905s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-9e161a0b-31ae-4137-a88d-2fe874c000d1 req-d64f7c89-1792-46bd-82f9-9be262891b43 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9e161a0b-31ae-4137-a88d-2fe874c000d1 req-d64f7c89-1792-46bd-82f9-9be262891b43 service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9e161a0b-31ae-4137-a88d-2fe874c000d1 req-d64f7c89-1792-46bd-82f9-9be262891b43 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9e161a0b-31ae-4137-a88d-2fe874c000d1 req-d64f7c89-1792-46bd-82f9-9be262891b43 service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-9e161a0b-31ae-4137-a88d-2fe874c000d1 req-d64f7c89-1792-46bd-82f9-9be262891b43 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:41 user nova-compute[71317]: WARNING nova.compute.manager [req-9e161a0b-31ae-4137-a88d-2fe874c000d1 req-d64f7c89-1792-46bd-82f9-9be262891b43 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state resized and task_state resize_reverting.
Apr 14 04:27:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for a0a20aeb-ed06-4c65-b87d-74f8f825457d due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:27:41 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446461.7501895, a0a20aeb-ed06-4c65-b87d-74f8f825457d => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:27:41 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] VM Resumed (Lifecycle Event)
Apr 14 04:27:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:27:41 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance running successfully.
Apr 14 04:27:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] finish_revert_migration finished successfully. {{(pid=71317) finish_revert_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11836}}
Apr 14 04:27:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:27:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: resized, current task_state: resize_reverting, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:27:41 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] During sync_power_state the instance has a pending task (resize_reverting). Skip.
Apr 14 04:27:41 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446461.752192, a0a20aeb-ed06-4c65-b87d-74f8f825457d => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:27:41 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] VM Started (Lifecycle Event)
Apr 14 04:27:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:27:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: resized, current task_state: resize_reverting, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:27:41 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] During sync_power_state the instance has a pending task (resize_reverting). Skip.
Apr 14 04:27:41 user nova-compute[71317]: INFO nova.compute.manager [None req-10ed2ee9-f91e-46e5-8c41-69ca39ca38da tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating instance to original state: 'active'
Apr 14 04:27:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:45 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:27:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances with incomplete migration  {{(pid=71317) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 04:27:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:51 user nova-compute[71317]: DEBUG nova.compute.manager [req-d573bb18-7dac-4b06-8c6a-b092ce20a414 req-cdc634ce-fa98-4283-bde8-1d46d15ea7e8 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-changed-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:51 user nova-compute[71317]: DEBUG nova.compute.manager [req-d573bb18-7dac-4b06-8c6a-b092ce20a414 req-cdc634ce-fa98-4283-bde8-1d46d15ea7e8 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Refreshing instance network info cache due to event network-changed-dc662cd5-f74a-40ee-bc14-e0948cbc4f36. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:27:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d573bb18-7dac-4b06-8c6a-b092ce20a414 req-cdc634ce-fa98-4283-bde8-1d46d15ea7e8 service nova] Acquiring lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:27:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d573bb18-7dac-4b06-8c6a-b092ce20a414 req-cdc634ce-fa98-4283-bde8-1d46d15ea7e8 service nova] Acquired lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:27:51 user nova-compute[71317]: DEBUG nova.network.neutron [req-d573bb18-7dac-4b06-8c6a-b092ce20a414 req-cdc634ce-fa98-4283-bde8-1d46d15ea7e8 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Refreshing network info cache for port dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:27:52 user nova-compute[71317]: DEBUG nova.network.neutron [req-d573bb18-7dac-4b06-8c6a-b092ce20a414 req-cdc634ce-fa98-4283-bde8-1d46d15ea7e8 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updated VIF entry in instance network info cache for port dc662cd5-f74a-40ee-bc14-e0948cbc4f36. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:27:52 user nova-compute[71317]: DEBUG nova.network.neutron [req-d573bb18-7dac-4b06-8c6a-b092ce20a414 req-cdc634ce-fa98-4283-bde8-1d46d15ea7e8 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updating instance_info_cache with network_info: [{"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:27:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d573bb18-7dac-4b06-8c6a-b092ce20a414 req-cdc634ce-fa98-4283-bde8-1d46d15ea7e8 service nova] Releasing lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:27:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:52 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lazy-loading 'flavor' on Instance uuid 4a1c695b-2ca4-4652-8fb6-19accdab6ebe {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:52 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Ignoring supplied device name: /dev/vdb
Apr 14 04:27:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.077s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:53 user nova-compute[71317]: INFO nova.compute.manager [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Attaching volume b9113785-28aa-46da-ae7d-d2f2e9a011b2 to /dev/vdb
Apr 14 04:27:53 user nova-compute[71317]: DEBUG os_brick.utils [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b225b478-4fef-4da3-9b64-cad2fa0ce10b]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b188988d-8f46-427d-822c-67cf621e7bb9]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1267320f-c922-4584-ab59-c560fa92f0ad]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[47c53603-e55c-4bed-bc7c-15e1d13384ac]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:27:53 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[59aa1641-333b-4d82-bf3c-91d42682b431]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG os_brick.utils [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] <== get_connector_properties: return (146ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Updating existing volume attachment record: 0f1d889c-5759-4004-a6ca-b682af8b390f {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:53 user nova-compute[71317]: INFO nova.compute.manager [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Terminating instance
Apr 14 04:27:53 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-3854aeb0-dfb4-45c4-8884-4e1441ab203f req-b4df2e95-8a55-44f2-adea-0eb424bfd138 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-changed-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-3854aeb0-dfb4-45c4-8884-4e1441ab203f req-b4df2e95-8a55-44f2-adea-0eb424bfd138 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Refreshing instance network info cache due to event network-changed-dc662cd5-f74a-40ee-bc14-e0948cbc4f36. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3854aeb0-dfb4-45c4-8884-4e1441ab203f req-b4df2e95-8a55-44f2-adea-0eb424bfd138 service nova] Acquiring lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3854aeb0-dfb4-45c4-8884-4e1441ab203f req-b4df2e95-8a55-44f2-adea-0eb424bfd138 service nova] Acquired lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG nova.network.neutron [req-3854aeb0-dfb4-45c4-8884-4e1441ab203f req-b4df2e95-8a55-44f2-adea-0eb424bfd138 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Refreshing network info cache for port dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:53 user nova-compute[71317]: INFO nova.compute.manager [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Detaching volume d8f57c49-602f-450f-8b9e-69f1f0255cee
Apr 14 04:27:53 user nova-compute[71317]: INFO nova.virt.block_device [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Attempting to driver detach volume d8f57c49-602f-450f-8b9e-69f1f0255cee from mountpoint /dev/vdb
Apr 14 04:27:53 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Attempting to detach device vdb from instance 5273d202-ffc5-43c6-bb4f-f6d1a6855271 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:27:53 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:27:53 user nova-compute[71317]:   <source dev="/dev/sde"/>
Apr 14 04:27:53 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:27:53 user nova-compute[71317]:   <serial>d8f57c49-602f-450f-8b9e-69f1f0255cee</serial>
Apr 14 04:27:53 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:27:53 user nova-compute[71317]: </disk>
Apr 14 04:27:53 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:27:53 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Successfully detached device vdb from instance 5273d202-ffc5-43c6-bb4f-f6d1a6855271 from the persistent domain config.
Apr 14 04:27:53 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 5273d202-ffc5-43c6-bb4f-f6d1a6855271 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:27:53 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:27:53 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:27:53 user nova-compute[71317]:   <source dev="/dev/sde"/>
Apr 14 04:27:53 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:27:53 user nova-compute[71317]:   <serial>d8f57c49-602f-450f-8b9e-69f1f0255cee</serial>
Apr 14 04:27:53 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:27:53 user nova-compute[71317]: </disk>
Apr 14 04:27:53 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-3854aeb0-dfb4-45c4-8884-4e1441ab203f req-b4df2e95-8a55-44f2-adea-0eb424bfd138 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updated VIF entry in instance network info cache for port dc662cd5-f74a-40ee-bc14-e0948cbc4f36. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-3854aeb0-dfb4-45c4-8884-4e1441ab203f req-b4df2e95-8a55-44f2-adea-0eb424bfd138 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updating instance_info_cache with network_info: [{"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3854aeb0-dfb4-45c4-8884-4e1441ab203f req-b4df2e95-8a55-44f2-adea-0eb424bfd138 service nova] Releasing lock "refresh_cache-5273d202-ffc5-43c6-bb4f-f6d1a6855271" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446474.446186, 5273d202-ffc5-43c6-bb4f-f6d1a6855271 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 5273d202-ffc5-43c6-bb4f-f6d1a6855271 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:27:54 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Successfully detached device vdb from instance 5273d202-ffc5-43c6-bb4f-f6d1a6855271 from the live domain config.
Apr 14 04:27:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HfrLsjmV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:27:54 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:27:54 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Instance destroyed successfully.
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lazy-loading 'resources' on Instance uuid aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:26:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1404624303',display_name='tempest-TestVolumeBootPattern-server-1404624303',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1404624303',id=26,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:26:22Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-oumzu04l',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:26:22Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8864dbfd-6219-4843-aa68-aea87851f10b", "address": "fa:16:3e:f4:d4:39", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8864dbfd-62", "ovs_interfaceid": "8864dbfd-6219-4843-aa68-aea87851f10b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "8864dbfd-6219-4843-aa68-aea87851f10b", "address": "fa:16:3e:f4:d4:39", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8864dbfd-62", "ovs_interfaceid": "8864dbfd-6219-4843-aa68-aea87851f10b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f4:d4:39,bridge_name='br-int',has_traffic_filtering=True,id=8864dbfd-6219-4843-aa68-aea87851f10b,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8864dbfd-62') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_vif [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f4:d4:39,bridge_name='br-int',has_traffic_filtering=True,id=8864dbfd-6219-4843-aa68-aea87851f10b,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8864dbfd-62') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fdd4808f-f89a-41f5-ba00-8d242098acf7]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:27:54 user nova-compute[71317]: No targets found.
Apr 14 04:27:54 user nova-compute[71317]: iSNS:
Apr 14 04:27:54 user nova-compute[71317]: No targets found.
Apr 14 04:27:54 user nova-compute[71317]: STATIC:
Apr 14 04:27:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:27:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f
Apr 14 04:27:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee
Apr 14 04:27:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:27:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4
Apr 14 04:27:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:27:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:54 user nova-compute[71317]: FIRMWARE:
Apr 14 04:27:54 user nova-compute[71317]: No targets found.
Apr 14 04:27:54 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:27:54 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:27:54 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8864dbfd-62, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:54 user nova-compute[71317]: INFO os_vif [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f4:d4:39,bridge_name='br-int',has_traffic_filtering=True,id=8864dbfd-6219-4843-aa68-aea87851f10b,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8864dbfd-62')
Apr 14 04:27:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PxcrKdyl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:27:54 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[74025f9d-d0bb-4e97-a5e7-e8d736461016]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c856ec4f-e048-4626-b234-8f74e8401291]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c43b5ea0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee'): ({'sde'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Removing single pathed devices sde {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[de4a57b1-198a-4097-a14f-a5db5ad1d93e]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e4577ee-b553-4388-9ab6-f1ccadd6c8c1]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Flushing IO for device /dev/sde {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[14497dad-ded9-43cc-ab58-c36f449e612c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.042s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0df19e4c-6371-452c-b4b1-2162566c0a8d]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed93100a-732b-4ae1-928e-3775765b14f5]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5818d64-8402-41d4-88e5-255b1b2f0742]: (4, ('Logging out of session [sid: 28, target: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee, portal: 172.16.0.220,3260]\nLogout of [sid: 28, target: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 28, target: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee, portal: 172.16.0.220,3260]
Apr 14 04:27:54 user nova-compute[71317]: Logout of [sid: 28, target: iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee, portal: 172.16.0.220,3260] successful.
Apr 14 04:27:54 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d8f57c49-602f-450f-8b9e-69f1f0255cee -p 172.16.0.220:3260 --op delete" returned: 0 in 0.037s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b8e18204-8478-4774-bf8a-873e449476ea]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.376s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] <== disconnect_volume: return (379ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.315s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[93e4ad33-677b-4506-9318-576d5e3987a1]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:27:54 user nova-compute[71317]: No targets found.
Apr 14 04:27:54 user nova-compute[71317]: iSNS:
Apr 14 04:27:54 user nova-compute[71317]: No targets found.
Apr 14 04:27:54 user nova-compute[71317]: STATIC:
Apr 14 04:27:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:27:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f
Apr 14 04:27:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:27:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4
Apr 14 04:27:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:27:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:27:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:27:54 user nova-compute[71317]: FIRMWARE:
Apr 14 04:27:54 user nova-compute[71317]: No targets found.
Apr 14 04:27:54 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:27:54 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:27:54 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[37d20f40-3b0c-483a-8118-d0efa600dc97]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[994f27c5-af1b-427e-b973-b3cef8fa4ae7]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:54 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d1809f30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a61f8ab6-e462-4d9c-8bcd-66850503946f]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3533945-5ba4-475d-b1a1-855b77217085]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb277a84-de83-4a4c-bd04-5c8be7b85f70]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.050s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[882cf15d-07e9-4859-b0cf-dd019faace7e]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba1f367f-d003-4abc-935b-df179101c8a5]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --logout" returned: 0 in 0.069s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[92efb703-329e-410e-bb3d-28e515a7b105]: (4, ('Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f, portal: 172.16.0.220,3260]\nLogout of [sid: 29, target: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f, portal: 172.16.0.220,3260]
Apr 14 04:27:55 user nova-compute[71317]: Logout of [sid: 29, target: iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f, portal: 172.16.0.220,3260] successful.
Apr 14 04:27:55 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:24291e20-0b7c-4b2c-9351-0e6cd072508f -p 172.16.0.220:3260 --op delete" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ebf33b3f-a0af-435c-b05b-81ebac7e010d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.379s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== disconnect_volume: return (695ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:27:55 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Deleting instance files /opt/stack/data/nova/instances/aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd_del
Apr 14 04:27:55 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Deletion of /opt/stack/data/nova/instances/aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd_del complete
Apr 14 04:27:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:55 user nova-compute[71317]: INFO nova.compute.manager [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Took 1.92 seconds to destroy the instance on the hypervisor.
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-f7325040-7b11-48be-b3fc-a7fb76a17af3 req-9539ddda-dfe5-4b9a-91c8-ad500fe44c2c service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Received event network-vif-unplugged-8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f7325040-7b11-48be-b3fc-a7fb76a17af3 req-9539ddda-dfe5-4b9a-91c8-ad500fe44c2c service nova] Acquiring lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f7325040-7b11-48be-b3fc-a7fb76a17af3 req-9539ddda-dfe5-4b9a-91c8-ad500fe44c2c service nova] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f7325040-7b11-48be-b3fc-a7fb76a17af3 req-9539ddda-dfe5-4b9a-91c8-ad500fe44c2c service nova] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-f7325040-7b11-48be-b3fc-a7fb76a17af3 req-9539ddda-dfe5-4b9a-91c8-ad500fe44c2c service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] No waiting events found dispatching network-vif-unplugged-8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-f7325040-7b11-48be-b3fc-a7fb76a17af3 req-9539ddda-dfe5-4b9a-91c8-ad500fe44c2c service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Received event network-vif-unplugged-8864dbfd-6219-4843-aa68-aea87851f10b for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-f7325040-7b11-48be-b3fc-a7fb76a17af3 req-9539ddda-dfe5-4b9a-91c8-ad500fe44c2c service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Received event network-vif-plugged-8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f7325040-7b11-48be-b3fc-a7fb76a17af3 req-9539ddda-dfe5-4b9a-91c8-ad500fe44c2c service nova] Acquiring lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f7325040-7b11-48be-b3fc-a7fb76a17af3 req-9539ddda-dfe5-4b9a-91c8-ad500fe44c2c service nova] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f7325040-7b11-48be-b3fc-a7fb76a17af3 req-9539ddda-dfe5-4b9a-91c8-ad500fe44c2c service nova] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-f7325040-7b11-48be-b3fc-a7fb76a17af3 req-9539ddda-dfe5-4b9a-91c8-ad500fe44c2c service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] No waiting events found dispatching network-vif-plugged-8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:55 user nova-compute[71317]: WARNING nova.compute.manager [req-f7325040-7b11-48be-b3fc-a7fb76a17af3 req-9539ddda-dfe5-4b9a-91c8-ad500fe44c2c service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Received unexpected event network-vif-plugged-8864dbfd-6219-4843-aa68-aea87851f10b for instance with vm_state active and task_state deleting.
Apr 14 04:27:55 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:27:55 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Took 0.64 seconds to deallocate network for instance.
Apr 14 04:27:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-9937cd1d-b599-4459-a121-36d2a6292ea9 req-47948fd9-aa85-41a7-988d-30bb8cce6e6e service nova] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Received event network-vif-deleted-8864dbfd-6219-4843-aa68-aea87851f10b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:56 user nova-compute[71317]: DEBUG nova.objects.instance [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'flavor' on Instance uuid 5273d202-ffc5-43c6-bb4f-f6d1a6855271 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-34f8a0d2-9f1d-4f63-b4f0-94db8b4ed107 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.773s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:56 user nova-compute[71317]: INFO nova.compute.manager [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Took 0.86 seconds to detach 1 volumes for instance.
Apr 14 04:27:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Deleting volume: 24291e20-0b7c-4b2c-9351-0e6cd072508f {{(pid=71317) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 14 04:27:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:56 user nova-compute[71317]: DEBUG nova.objects.instance [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'flavor' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.053s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:57 user nova-compute[71317]: INFO nova.compute.manager [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attaching volume babb4943-f427-4d60-8c85-091fdae699ee to /dev/vdb
Apr 14 04:27:57 user nova-compute[71317]: DEBUG os_brick.utils [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[da4ded60-45dc-4eb7-8096-ad47ac84a08a]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0b5bc20-d297-4373-b6da-89c96e16a9a4]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8baae2c-d96e-40be-bccb-772d5b9b1ece]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0c26935-75e9-4058-9db4-f01dbe2da589]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:27:57 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2bf86fb0-0992-4ff3-ae4c-08e35403aa45]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG os_brick.utils [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] <== get_connector_properties: return (140ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Updating existing volume attachment record: e018d31b-6aa2-4a57-a91f-3b5c58c9a342 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.474s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:57 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Deleted allocations for instance aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7c48dc7-3212-4b47-9372-14dd06b974f2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.199s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:57 user nova-compute[71317]: INFO nova.compute.manager [None req-8912674f-88e2-4e5c-b091-828f1c641148 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Unrescuing
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8912674f-88e2-4e5c-b091-828f1c641148 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8912674f-88e2-4e5c-b091-828f1c641148 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquired lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:27:57 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8912674f-88e2-4e5c-b091-828f1c641148 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'IWZgAHoj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:27:58 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:27:58 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:27:58 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[87aa828c-0bc8-458b-bc55-a314b44a6a46]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:27:58 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a09d3e7f-12c2-46a9-9575-e913fe2478ee]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2] added
Apr 14 04:27:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[37935af5-5f38-4985-8fd4-a1b2d7899bc4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[625e2d8f-98b0-46d2-9e88-75ebb7a8445a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v IWZgAHoj {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v IWZgAHoj" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2851057-de07-4b13-9a1b-69fe21733ba7]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'IWZgAHoj'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f15e6f53-8c0e-4903-a5b7-6ae9eca6f9f7]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b4c3396-66c7-435d-8033-f5b6f379dbfe]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8912674f-88e2-4e5c-b091-828f1c641148 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updating instance_info_cache with network_info: [{"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8912674f-88e2-4e5c-b091-828f1c641148 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Releasing lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG nova.objects.instance [None req-8912674f-88e2-4e5c-b091-828f1c641148 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'flavor' on Instance uuid 3df97db3-fd6c-4be6-af32-7cde9dbcce75 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --login" returned: 0 in 0.095s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[297d15a8-2e0c-4075-b22f-73541f8b7bc4]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2, portal: 172.16.0.220,3260]
Apr 14 04:27:58 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2, portal: 172.16.0.220,3260] successful.
Apr 14 04:27:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7da72fde-456a-41a9-8cbf-4d01adc1e473]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[edbada56-3d62-446e-9468-a8dd1a7f98c8]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)
Apr 14 04:27:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.335s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] HCTL ('34', '-', '-', 0) found on session 35 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1168557b-3014-44aa-8d36-6a18aaa2bf9c]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Searching for a device in session 35 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'IWZgAHoj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-bdffe061-46a9-402b-bdaf-c4ad9416f1c2 req-8b7828e7-e6e8-4ab3-bc21-2205d790abd1 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-unplugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bdffe061-46a9-402b-bdaf-c4ad9416f1c2 req-8b7828e7-e6e8-4ab3-bc21-2205d790abd1 service nova] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bdffe061-46a9-402b-bdaf-c4ad9416f1c2 req-8b7828e7-e6e8-4ab3-bc21-2205d790abd1 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bdffe061-46a9-402b-bdaf-c4ad9416f1c2 req-8b7828e7-e6e8-4ab3-bc21-2205d790abd1 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:27:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-bdffe061-46a9-402b-bdaf-c4ad9416f1c2 req-8b7828e7-e6e8-4ab3-bc21-2205d790abd1 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] No waiting events found dispatching network-vif-unplugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:27:58 user nova-compute[71317]: WARNING nova.compute.manager [req-bdffe061-46a9-402b-bdaf-c4ad9416f1c2 req-8b7828e7-e6e8-4ab3-bc21-2205d790abd1 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received unexpected event network-vif-unplugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:27:59 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Instance destroyed successfully.
Apr 14 04:27:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:27:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.375s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:27:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] <== connect_volume: return (1377ms) {'type': 'block', 'scsi_wwn': '23636313935613536', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:27:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23636313935613536', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:27:59 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lazy-loading 'flavor' on Instance uuid 4a1c695b-2ca4-4652-8fb6-19accdab6ebe {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:27:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:27:59 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:27:59 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:27:59 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:27:59 user nova-compute[71317]:   <serial>b9113785-28aa-46da-ae7d-d2f2e9a011b2</serial>
Apr 14 04:27:59 user nova-compute[71317]: </disk>
Apr 14 04:27:59 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] No VIF found with MAC fa:16:3e:5b:1b:7f, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ef5f379e-b7e6-4f4c-b73a-00c431daa984 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.381s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-68d2ee88-f604-4a0c-9332-02157a5dd52d req-cf6ad0ec-63b3-4fd7-9c38-330909a30174 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-68d2ee88-f604-4a0c-9332-02157a5dd52d req-cf6ad0ec-63b3-4fd7-9c38-330909a30174 service nova] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-68d2ee88-f604-4a0c-9332-02157a5dd52d req-cf6ad0ec-63b3-4fd7-9c38-330909a30174 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-68d2ee88-f604-4a0c-9332-02157a5dd52d req-cf6ad0ec-63b3-4fd7-9c38-330909a30174 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-68d2ee88-f604-4a0c-9332-02157a5dd52d req-cf6ad0ec-63b3-4fd7-9c38-330909a30174 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] No waiting events found dispatching network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:28:00 user nova-compute[71317]: WARNING nova.compute.manager [req-68d2ee88-f604-4a0c-9332-02157a5dd52d req-cf6ad0ec-63b3-4fd7-9c38-330909a30174 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received unexpected event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:28:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wXCvrjzG', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:28:02 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:28:02 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:28:02 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a4b0972-fb43-4f78-84aa-30e71e765b05]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:28:02 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[17cc6dc8-9788-4bf1-a430-4a8f445234b2]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee] added
Apr 14 04:28:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb4669b0-8539-4095-ab77-8c6ec9cf15f2]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[31ed439e-6420-4f63-8a3e-dab14e544d16]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wXCvrjzG {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wXCvrjzG" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9cf1b34-0bec-4755-9292-6db2823a69b0]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'wXCvrjzG'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.039s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[577d6241-f44e-411d-bd43-7fbee8631221]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[11795376-88cd-42f7-b2a3-da3287826364]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --login" returned: 0 in 0.135s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b03edb57-a246-455a-add5-d7e2a332b00a]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee, portal: 172.16.0.220,3260]
Apr 14 04:28:02 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee, portal: 172.16.0.220,3260] successful.
Apr 14 04:28:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.093s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e76ace90-c397-4565-9384-0153bcbcb204]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[da5c64e6-4d04-4c68-b4c4-77ad78da23d2]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee (non-flash)
Apr 14 04:28:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)
Apr 14 04:28:02 user nova-compute[71317]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee (non-flash)
Apr 14 04:28:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.470s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] HCTL ('36', '-', '-', 0) found on session 36 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[00a167f9-441f-4fb6-a9c3-3a73296ad844]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Searching for a device in session 36 and hctl ['36', '*', '*', 0] yield: None {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.compute.manager [req-82901559-c3b2-4b13-adf0-20b01f6d1c6d req-653b1eec-faee-4292-a0d3-f7a2ef3fc6cd service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-82901559-c3b2-4b13-adf0-20b01f6d1c6d req-653b1eec-faee-4292-a0d3-f7a2ef3fc6cd service nova] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-82901559-c3b2-4b13-adf0-20b01f6d1c6d req-653b1eec-faee-4292-a0d3-f7a2ef3fc6cd service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-82901559-c3b2-4b13-adf0-20b01f6d1c6d req-653b1eec-faee-4292-a0d3-f7a2ef3fc6cd service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.compute.manager [req-82901559-c3b2-4b13-adf0-20b01f6d1c6d req-653b1eec-faee-4292-a0d3-f7a2ef3fc6cd service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] No waiting events found dispatching network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:28:03 user nova-compute[71317]: WARNING nova.compute.manager [req-82901559-c3b2-4b13-adf0-20b01f6d1c6d req-653b1eec-faee-4292-a0d3-f7a2ef3fc6cd service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received unexpected event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.compute.manager [req-82901559-c3b2-4b13-adf0-20b01f6d1c6d req-653b1eec-faee-4292-a0d3-f7a2ef3fc6cd service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-82901559-c3b2-4b13-adf0-20b01f6d1c6d req-653b1eec-faee-4292-a0d3-f7a2ef3fc6cd service nova] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-82901559-c3b2-4b13-adf0-20b01f6d1c6d req-653b1eec-faee-4292-a0d3-f7a2ef3fc6cd service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-82901559-c3b2-4b13-adf0-20b01f6d1c6d req-653b1eec-faee-4292-a0d3-f7a2ef3fc6cd service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.compute.manager [req-82901559-c3b2-4b13-adf0-20b01f6d1c6d req-653b1eec-faee-4292-a0d3-f7a2ef3fc6cd service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] No waiting events found dispatching network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:28:03 user nova-compute[71317]: WARNING nova.compute.manager [req-82901559-c3b2-4b13-adf0-20b01f6d1c6d req-653b1eec-faee-4292-a0d3-f7a2ef3fc6cd service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received unexpected event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:28:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for 3df97db3-fd6c-4be6-af32-7cde9dbcce75 due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446483.2903705, 3df97db3-fd6c-4be6-af32-7cde9dbcce75 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:28:03 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] VM Resumed (Lifecycle Event)
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8912674f-88e2-4e5c-b091-828f1c641148 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:28:03 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446483.2953362, 3df97db3-fd6c-4be6-af32-7cde9dbcce75 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:28:03 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] VM Started (Lifecycle Event)
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Searching for a device in session 36 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wXCvrjzG', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.500s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] <== connect_volume: return (1501ms) {'type': 'block', 'scsi_wwn': '23935333538326666', 'path': '/dev/sde'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23935333538326666', 'path': '/dev/sde'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.objects.instance [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'flavor' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:28:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:28:03 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:28:03 user nova-compute[71317]:   <source dev="/dev/sde"/>
Apr 14 04:28:03 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:28:03 user nova-compute[71317]:   <serial>babb4943-f427-4d60-8c85-091fdae699ee</serial>
Apr 14 04:28:03 user nova-compute[71317]: </disk>
Apr 14 04:28:03 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:28:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:28:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:28:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No VIF found with MAC fa:16:3e:17:59:61, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:28:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b472a2a2-973f-4d90-b27f-8b41b3325dc3 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.131s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:05 user nova-compute[71317]: INFO nova.compute.manager [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Detaching volume babb4943-f427-4d60-8c85-091fdae699ee
Apr 14 04:28:05 user nova-compute[71317]: INFO nova.virt.block_device [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attempting to driver detach volume babb4943-f427-4d60-8c85-091fdae699ee from mountpoint /dev/vdb
Apr 14 04:28:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Attempting to detach device vdb from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:28:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:28:05 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:28:05 user nova-compute[71317]:   <source dev="/dev/sde"/>
Apr 14 04:28:05 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:28:05 user nova-compute[71317]:   <serial>babb4943-f427-4d60-8c85-091fdae699ee</serial>
Apr 14 04:28:05 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:28:05 user nova-compute[71317]: </disk>
Apr 14 04:28:05 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:28:05 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Successfully detached device vdb from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the persistent domain config.
Apr 14 04:28:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:28:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:28:05 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:28:05 user nova-compute[71317]:   <source dev="/dev/sde"/>
Apr 14 04:28:05 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:28:05 user nova-compute[71317]:   <serial>babb4943-f427-4d60-8c85-091fdae699ee</serial>
Apr 14 04:28:05 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:28:05 user nova-compute[71317]: </disk>
Apr 14 04:28:05 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446486.047857, f593ceb0-de44-490b-b365-3a05a3acd066 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:28:06 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Successfully detached device vdb from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the live domain config.
Apr 14 04:28:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wXCvrjzG', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:28:06 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbdf5ef1-a348-4f9d-8f9d-abd88c5012c2]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:28:06 user nova-compute[71317]: No targets found.
Apr 14 04:28:06 user nova-compute[71317]: iSNS:
Apr 14 04:28:06 user nova-compute[71317]: No targets found.
Apr 14 04:28:06 user nova-compute[71317]: STATIC:
Apr 14 04:28:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2
Apr 14 04:28:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:28:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee
Apr 14 04:28:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:28:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4
Apr 14 04:28:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:28:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:06 user nova-compute[71317]: FIRMWARE:
Apr 14 04:28:06 user nova-compute[71317]: No targets found.
Apr 14 04:28:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:28:06 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:28:06 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcfe9ad3-502d-4311-9402-60f177a3c845]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9bbbb674-ae37-4b17-8e1c-7df57bc1aa26]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:06 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:28:06 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:06 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:06 user nova-compute[71317]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)
Apr 14 04:28:06 user nova-compute[71317]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee (non-flash)
Apr 14 04:28:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:06 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:28:06 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:06 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:06 user nova-compute[71317]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)
Apr 14 04:28:06 user nova-compute[71317]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee (non-flash)
Apr 14 04:28:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84b4134d30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee'): ({'sde'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Removing single pathed devices sde {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab24e087-f6e8-49fc-834a-c77af49c8acc]: (4, ('path checker states:\nup                  5\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3500a4b7-d4c4-40d2-98e2-12ae346e83c3]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Flushing IO for device /dev/sde {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b6a6b63-1852-4658-aa6c-2d72aa5a7aea]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.046s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e28cffab-9b08-4b2e-927c-7ab161cddbb0]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9680ebed-2053-40bf-9b0d-03038d009585]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b04bbcb-7f04-46e6-aee6-06f33aa1c8e6]: (4, ('Logging out of session [sid: 36, target: iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee, portal: 172.16.0.220,3260]\nLogout of [sid: 36, target: iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 36, target: iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee, portal: 172.16.0.220,3260]
Apr 14 04:28:06 user nova-compute[71317]: Logout of [sid: 36, target: iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee, portal: 172.16.0.220,3260] successful.
Apr 14 04:28:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:babb4943-f427-4d60-8c85-091fdae699ee -p 172.16.0.220:3260 --op delete" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[668ef978-bdf5-441b-9a37-31e8dba48d1c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.332s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] <== disconnect_volume: return (333ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:28:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:28:07 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'flavor' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:28:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a0d37cbc-e6a4-4b73-83fb-b1f6705982c4 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.709s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:09 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446474.453436, aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:28:09 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] VM Stopped (Lifecycle Event)
Apr 14 04:28:09 user nova-compute[71317]: DEBUG nova.compute.manager [None req-419ac763-5d54-4943-8fd5-259acb4fa7f3 None None] [instance: aa8312da-d5e6-4ca7-a7b0-e11ed3351bdd] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:28:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-90f051ad-6d0e-4b11-99ba-61b25b852e81 req-2324e09e-555f-425e-a838-d5157d70b3d3 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-changed-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-90f051ad-6d0e-4b11-99ba-61b25b852e81 req-2324e09e-555f-425e-a838-d5157d70b3d3 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Refreshing instance network info cache due to event network-changed-61ec63cd-2733-40c4-a8c6-9243a6a3c144. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:28:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-90f051ad-6d0e-4b11-99ba-61b25b852e81 req-2324e09e-555f-425e-a838-d5157d70b3d3 service nova] Acquiring lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:28:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-90f051ad-6d0e-4b11-99ba-61b25b852e81 req-2324e09e-555f-425e-a838-d5157d70b3d3 service nova] Acquired lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:28:10 user nova-compute[71317]: DEBUG nova.network.neutron [req-90f051ad-6d0e-4b11-99ba-61b25b852e81 req-2324e09e-555f-425e-a838-d5157d70b3d3 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Refreshing network info cache for port 61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:28:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" acquired by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:11 user nova-compute[71317]: INFO nova.compute.manager [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Shelve offloading
Apr 14 04:28:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:11 user nova-compute[71317]: DEBUG nova.network.neutron [req-90f051ad-6d0e-4b11-99ba-61b25b852e81 req-2324e09e-555f-425e-a838-d5157d70b3d3 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updated VIF entry in instance network info cache for port 61ec63cd-2733-40c4-a8c6-9243a6a3c144. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:28:11 user nova-compute[71317]: DEBUG nova.network.neutron [req-90f051ad-6d0e-4b11-99ba-61b25b852e81 req-2324e09e-555f-425e-a838-d5157d70b3d3 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updating instance_info_cache with network_info: [{"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:28:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-90f051ad-6d0e-4b11-99ba-61b25b852e81 req-2324e09e-555f-425e-a838-d5157d70b3d3 service nova] Releasing lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:28:11 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Instance destroyed successfully.
Apr 14 04:28:11 user nova-compute[71317]: DEBUG nova.compute.manager [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:28:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:28:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquired lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:28:11 user nova-compute[71317]: DEBUG nova.network.neutron [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:28:12 user nova-compute[71317]: DEBUG nova.network.neutron [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updating instance_info_cache with network_info: [{"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:28:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Releasing lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:28:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-5ec41c77-2332-46dc-a09f-b3e5ae5e852f req-df693537-6f3a-4d3d-8f93-b99df0b1f9f7 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-vif-unplugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5ec41c77-2332-46dc-a09f-b3e5ae5e852f req-df693537-6f3a-4d3d-8f93-b99df0b1f9f7 service nova] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5ec41c77-2332-46dc-a09f-b3e5ae5e852f req-df693537-6f3a-4d3d-8f93-b99df0b1f9f7 service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5ec41c77-2332-46dc-a09f-b3e5ae5e852f req-df693537-6f3a-4d3d-8f93-b99df0b1f9f7 service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-5ec41c77-2332-46dc-a09f-b3e5ae5e852f req-df693537-6f3a-4d3d-8f93-b99df0b1f9f7 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] No waiting events found dispatching network-vif-unplugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:28:12 user nova-compute[71317]: WARNING nova.compute.manager [req-5ec41c77-2332-46dc-a09f-b3e5ae5e852f req-df693537-6f3a-4d3d-8f93-b99df0b1f9f7 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received unexpected event network-vif-unplugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 for instance with vm_state active and task_state shelving.
Apr 14 04:28:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-5ec41c77-2332-46dc-a09f-b3e5ae5e852f req-df693537-6f3a-4d3d-8f93-b99df0b1f9f7 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5ec41c77-2332-46dc-a09f-b3e5ae5e852f req-df693537-6f3a-4d3d-8f93-b99df0b1f9f7 service nova] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5ec41c77-2332-46dc-a09f-b3e5ae5e852f req-df693537-6f3a-4d3d-8f93-b99df0b1f9f7 service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5ec41c77-2332-46dc-a09f-b3e5ae5e852f req-df693537-6f3a-4d3d-8f93-b99df0b1f9f7 service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-5ec41c77-2332-46dc-a09f-b3e5ae5e852f req-df693537-6f3a-4d3d-8f93-b99df0b1f9f7 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] No waiting events found dispatching network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:28:12 user nova-compute[71317]: WARNING nova.compute.manager [req-5ec41c77-2332-46dc-a09f-b3e5ae5e852f req-df693537-6f3a-4d3d-8f93-b99df0b1f9f7 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received unexpected event network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 for instance with vm_state active and task_state shelving.
Apr 14 04:28:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Instance destroyed successfully.
Apr 14 04:28:13 user nova-compute[71317]: DEBUG nova.objects.instance [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lazy-loading 'resources' on Instance uuid 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:26:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-2065386753',display_name='tempest-TestShelveInstance-server-2065386753',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-2065386753',id=28,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGGk9hQ8wpWxQD87wrkXLIqSgBhES95isERnHNc/wFOMwogmDvk0s+HHEuuYbkzB2pEyv6u9qIhQ2bQkxjPV0WSr048nFyVVwxTOQQiEcmHX8pNahIhzR1Wk0Uv6MFb63A==',key_name='tempest-TestShelveInstance-2060150193',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:26:32Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='b0e15d3d3e0943c4b6bd54abf2de5fee',ramdisk_id='',reservation_id='r-osj4jtne',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1458140851',owner_user_name='tempest-TestShelveInstance-1458140851-project-member'},tags=<?>,task_state='shelving',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:26:32Z,user_data=None,user_id='36b6981092424de781e8403946f956d3',uuid=6a510dd3-bb77-4f6d-bdfd-180ce1d31c71,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Converting VIF {"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_vif [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap61ec63cd-27, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:28:13 user nova-compute[71317]: INFO os_vif [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27')
Apr 14 04:28:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-6e826a50-e75a-46d0-bb59-24f5b6b51bf2 req-90304a74-ccb0-40b5-b2ca-6769e5ee705a service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-changed-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-6e826a50-e75a-46d0-bb59-24f5b6b51bf2 req-90304a74-ccb0-40b5-b2ca-6769e5ee705a service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Refreshing instance network info cache due to event network-changed-61ec63cd-2733-40c4-a8c6-9243a6a3c144. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6e826a50-e75a-46d0-bb59-24f5b6b51bf2 req-90304a74-ccb0-40b5-b2ca-6769e5ee705a service nova] Acquiring lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6e826a50-e75a-46d0-bb59-24f5b6b51bf2 req-90304a74-ccb0-40b5-b2ca-6769e5ee705a service nova] Acquired lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG nova.network.neutron [req-6e826a50-e75a-46d0-bb59-24f5b6b51bf2 req-90304a74-ccb0-40b5-b2ca-6769e5ee705a service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Refreshing network info cache for port 61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pRhHtNJB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:28:13 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e97896c-a312-4815-920e-405359a15e31]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:28:13 user nova-compute[71317]: No targets found.
Apr 14 04:28:13 user nova-compute[71317]: iSNS:
Apr 14 04:28:13 user nova-compute[71317]: No targets found.
Apr 14 04:28:13 user nova-compute[71317]: STATIC:
Apr 14 04:28:13 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2
Apr 14 04:28:13 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:13 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:13 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:28:13 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:13 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:13 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:28:13 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:13 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:13 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4
Apr 14 04:28:13 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:13 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:13 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:28:13 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:13 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:13 user nova-compute[71317]: FIRMWARE:
Apr 14 04:28:13 user nova-compute[71317]: No targets found.
Apr 14 04:28:13 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:28:13 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:28:13 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[477eee42-aa44-4506-af00-68c5a052dd6b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[57390f36-e6d6-4ee2-a747-944902ea077a]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:13 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:28:13 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:13 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:13 user nova-compute[71317]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)
Apr 14 04:28:13 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:13 user nova-compute[71317]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:28:13 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:13 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:13 user nova-compute[71317]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)
Apr 14 04:28:13 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c43b5360>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4'): ({'sdf'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Removing single pathed devices sdf {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8f3e649-b133-4bb0-9622-f806bddcaed5]: (4, ('path checker states:\nup                  4\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3f4fe16-ec9a-4914-81c8-0165c7c01fbd]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Flushing IO for device /dev/sdf {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2df8123f-8b4b-4d32-9640-f6cbfa4e0415]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.043s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d1737e3-a7aa-46d5-a230-e89bff04d557]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[185b87e1-e9a5-4826-ad7d-6c349b0d5031]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --logout" returned: 0 in 0.066s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5f0f11c-06ab-4899-b734-7fd34d1f9072]: (4, ('Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260]\nLogout of [sid: 30, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260]
Apr 14 04:28:13 user nova-compute[71317]: Logout of [sid: 30, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260] successful.
Apr 14 04:28:13 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.127s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[02696642-23f3-4d24-b23c-b56561a05164]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.420s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] <== disconnect_volume: return (421ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:28:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:28:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Deleting instance files /opt/stack/data/nova/instances/6a510dd3-bb77-4f6d-bdfd-180ce1d31c71_del
Apr 14 04:28:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Deletion of /opt/stack/data/nova/instances/6a510dd3-bb77-4f6d-bdfd-180ce1d31c71_del complete
Apr 14 04:28:14 user nova-compute[71317]: DEBUG nova.network.neutron [req-6e826a50-e75a-46d0-bb59-24f5b6b51bf2 req-90304a74-ccb0-40b5-b2ca-6769e5ee705a service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updated VIF entry in instance network info cache for port 61ec63cd-2733-40c4-a8c6-9243a6a3c144. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:28:14 user nova-compute[71317]: DEBUG nova.network.neutron [req-6e826a50-e75a-46d0-bb59-24f5b6b51bf2 req-90304a74-ccb0-40b5-b2ca-6769e5ee705a service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updating instance_info_cache with network_info: [{"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": null, "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tap61ec63cd-27", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:28:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6e826a50-e75a-46d0-bb59-24f5b6b51bf2 req-90304a74-ccb0-40b5-b2ca-6769e5ee705a service nova] Releasing lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:28:14 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Deleted allocations for instance 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71
Apr 14 04:28:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:14 user nova-compute[71317]: DEBUG nova.objects.instance [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lazy-loading 'numa_topology' on Instance uuid 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:28:15 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:28:15 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:28:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.338s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b4f707de-ae1c-47af-8554-49fd3aefd32b tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" "released" by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" :: held 4.289s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:17 user nova-compute[71317]: INFO nova.compute.manager [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Detaching volume b9113785-28aa-46da-ae7d-d2f2e9a011b2
Apr 14 04:28:17 user nova-compute[71317]: INFO nova.virt.block_device [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Attempting to driver detach volume b9113785-28aa-46da-ae7d-d2f2e9a011b2 from mountpoint /dev/vdb
Apr 14 04:28:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Attempting to detach device vdb from instance 4a1c695b-2ca4-4652-8fb6-19accdab6ebe from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:28:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:28:17 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:28:17 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:28:17 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:28:17 user nova-compute[71317]:   <serial>b9113785-28aa-46da-ae7d-d2f2e9a011b2</serial>
Apr 14 04:28:17 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:28:17 user nova-compute[71317]: </disk>
Apr 14 04:28:17 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:28:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Successfully detached device vdb from instance 4a1c695b-2ca4-4652-8fb6-19accdab6ebe from the persistent domain config.
Apr 14 04:28:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 4a1c695b-2ca4-4652-8fb6-19accdab6ebe from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:28:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:28:17 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:28:17 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:28:17 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:28:17 user nova-compute[71317]:   <serial>b9113785-28aa-46da-ae7d-d2f2e9a011b2</serial>
Apr 14 04:28:17 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:28:17 user nova-compute[71317]: </disk>
Apr 14 04:28:17 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:28:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446498.1170888, 4a1c695b-2ca4-4652-8fb6-19accdab6ebe => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 4a1c695b-2ca4-4652-8fb6-19accdab6ebe {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:28:18 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Successfully detached device vdb from instance 4a1c695b-2ca4-4652-8fb6-19accdab6ebe from the live domain config.
Apr 14 04:28:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'IWZgAHoj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:28:18 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ee78b95-1fa5-4651-ad79-81e6dc8862d7]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:28:18 user nova-compute[71317]: No targets found.
Apr 14 04:28:18 user nova-compute[71317]: iSNS:
Apr 14 04:28:18 user nova-compute[71317]: No targets found.
Apr 14 04:28:18 user nova-compute[71317]: STATIC:
Apr 14 04:28:18 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2
Apr 14 04:28:18 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:18 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:18 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:28:18 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:18 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:18 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:28:18 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:18 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:18 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:28:18 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:18 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:18 user nova-compute[71317]: FIRMWARE:
Apr 14 04:28:18 user nova-compute[71317]: No targets found.
Apr 14 04:28:18 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:28:18 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:28:18 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0461e321-b5c9-481b-ae9f-1c4177210118]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a5a4811-4927-46dc-ac5c-265f058bc556]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:18 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:18 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:18 user nova-compute[71317]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)
Apr 14 04:28:18 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:18 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:18 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:18 user nova-compute[71317]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 (non-flash)
Apr 14 04:28:18 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84b4134af0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[92340854-0ffd-486a-856f-4cb32d9d17f4]: (4, ('path checker states:\nup                  3\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb32ba72-0693-45d9-9a37-5178e5701c64]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bec6ef27-abc3-4f21-b95c-9fc591f6b9b3]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.062s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4aa7f3b-9078-4492-b1e3-3ffc3dbdeff6]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4b66e6c-ee05-46e6-b5de-7a97579a50e7]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --logout" returned: 0 in 0.056s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c846e5a-b780-4600-8637-3e84c14ad935]: (4, ('Logging out of session [sid: 35, target: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2, portal: 172.16.0.220,3260]\nLogout of [sid: 35, target: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 35, target: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2, portal: 172.16.0.220,3260]
Apr 14 04:28:18 user nova-compute[71317]: Logout of [sid: 35, target: iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2, portal: 172.16.0.220,3260] successful.
Apr 14 04:28:18 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[77615b38-ee6c-4639-afe0-209301538bf0]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.311s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] <== disconnect_volume: return (313ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:28:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG nova.objects.instance [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lazy-loading 'flavor' on Instance uuid 4a1c695b-2ca4-4652-8fb6-19accdab6ebe {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b52986d0-2b12-488b-ad7b-ae07ddc568f9 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.671s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 1.567s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:19 user nova-compute[71317]: INFO nova.compute.manager [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Terminating instance
Apr 14 04:28:19 user nova-compute[71317]: DEBUG nova.compute.manager [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-306ae5ee-c61e-4776-9516-d21d0ff63ced req-037470eb-e477-4442-ba24-14815ca04262 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Received event network-vif-unplugged-260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-306ae5ee-c61e-4776-9516-d21d0ff63ced req-037470eb-e477-4442-ba24-14815ca04262 service nova] Acquiring lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-306ae5ee-c61e-4776-9516-d21d0ff63ced req-037470eb-e477-4442-ba24-14815ca04262 service nova] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-306ae5ee-c61e-4776-9516-d21d0ff63ced req-037470eb-e477-4442-ba24-14815ca04262 service nova] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-306ae5ee-c61e-4776-9516-d21d0ff63ced req-037470eb-e477-4442-ba24-14815ca04262 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] No waiting events found dispatching network-vif-unplugged-260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:28:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-306ae5ee-c61e-4776-9516-d21d0ff63ced req-037470eb-e477-4442-ba24-14815ca04262 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Received event network-vif-unplugged-260596e7-554d-4e86-b146-fe487c284c83 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:28:20 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Instance destroyed successfully.
Apr 14 04:28:20 user nova-compute[71317]: DEBUG nova.objects.instance [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lazy-loading 'resources' on Instance uuid 4a1c695b-2ca4-4652-8fb6-19accdab6ebe {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:26:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-693983371',display_name='tempest-VolumesSnapshotTestJSON-instance-693983371',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-693983371',id=27,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:26:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0848c9a67e2746e5b12b73920215b729',ramdisk_id='',reservation_id='r-co7qa03q',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesSnapshotTestJSON-1226436274',owner_user_name='tempest-VolumesSnapshotTestJSON-1226436274-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:26:13Z,user_data=None,user_id='e4a70281d6734c15b3c4b1b00e4b5d44',uuid=4a1c695b-2ca4-4652-8fb6-19accdab6ebe,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "260596e7-554d-4e86-b146-fe487c284c83", "address": "fa:16:3e:5b:1b:7f", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap260596e7-55", "ovs_interfaceid": "260596e7-554d-4e86-b146-fe487c284c83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Converting VIF {"id": "260596e7-554d-4e86-b146-fe487c284c83", "address": "fa:16:3e:5b:1b:7f", "network": {"id": "21178b4b-c88e-43e2-9154-d5116a59802d", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1183770515-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0848c9a67e2746e5b12b73920215b729", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap260596e7-55", "ovs_interfaceid": "260596e7-554d-4e86-b146-fe487c284c83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5b:1b:7f,bridge_name='br-int',has_traffic_filtering=True,id=260596e7-554d-4e86-b146-fe487c284c83,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap260596e7-55') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_vif [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5b:1b:7f,bridge_name='br-int',has_traffic_filtering=True,id=260596e7-554d-4e86-b146-fe487c284c83,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap260596e7-55') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap260596e7-55, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:28:20 user nova-compute[71317]: INFO os_vif [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5b:1b:7f,bridge_name='br-int',has_traffic_filtering=True,id=260596e7-554d-4e86-b146-fe487c284c83,network=Network(21178b4b-c88e-43e2-9154-d5116a59802d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap260596e7-55')
Apr 14 04:28:20 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'IWZgAHoj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:28:20 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c200a96-e778-4f73-b6cc-cccfb8aacf1d]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:28:20 user nova-compute[71317]: No targets found.
Apr 14 04:28:20 user nova-compute[71317]: iSNS:
Apr 14 04:28:20 user nova-compute[71317]: No targets found.
Apr 14 04:28:20 user nova-compute[71317]: STATIC:
Apr 14 04:28:20 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:28:20 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:20 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:20 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:28:20 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:20 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:20 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:28:20 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:28:20 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:28:20 user nova-compute[71317]: FIRMWARE:
Apr 14 04:28:20 user nova-compute[71317]: No targets found.
Apr 14 04:28:20 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:28:20 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:28:20 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b9113785-28aa-46da-ae7d-d2f2e9a011b2', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c4e0c65-cab4-49fb-bcb1-480914aca8f7]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2a61ed4-ac35-4e4e-9978-6ffed66dd221]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:20 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:20 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:20 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:20 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:20 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:20 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84b41340d0>, {}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Disconnecting from: [] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.076s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] <== disconnect_volume: return (78ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:28:20 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Deleting instance files /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe_del
Apr 14 04:28:20 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Deletion of /opt/stack/data/nova/instances/4a1c695b-2ca4-4652-8fb6-19accdab6ebe_del complete
Apr 14 04:28:20 user nova-compute[71317]: INFO nova.compute.manager [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Took 0.94 seconds to destroy the instance on the hypervisor.
Apr 14 04:28:20 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:28:20 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Took 0.53 seconds to deallocate network for instance.
Apr 14 04:28:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-197c28db-2736-4075-9daf-ed9becad42b4 req-7ec4205a-167b-4df2-8589-e5a84bc7069f service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Received event network-vif-deleted-260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:20 user nova-compute[71317]: WARNING nova.volume.cinder [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Attachment 0f1d889c-5759-4004-a6ca-b682af8b390f does not exist. Ignoring.: cinderclient.exceptions.NotFound: Volume attachment could not be found with filter: attachment_id = 0f1d889c-5759-4004-a6ca-b682af8b390f. (HTTP 404) (Request-ID: req-8911af93-a0ce-480b-942b-50f422fff12f)
Apr 14 04:28:20 user nova-compute[71317]: INFO nova.compute.manager [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Took 0.02 seconds to detach 1 volumes for instance.
Apr 14 04:28:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:21 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:28:21 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:28:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.308s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:21 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Deleted allocations for instance 4a1c695b-2ca4-4652-8fb6-19accdab6ebe
Apr 14 04:28:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33c9f5af-aa2a-409b-b1f2-9214042a00c2 tempest-VolumesSnapshotTestJSON-1226436274 tempest-VolumesSnapshotTestJSON-1226436274-project-member] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.962s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-f7c956c4-e7b2-4557-badc-fd4e665bc8b9 req-bb17b551-fa48-499c-86b3-ef791bb4cbd8 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Received event network-vif-plugged-260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f7c956c4-e7b2-4557-badc-fd4e665bc8b9 req-bb17b551-fa48-499c-86b3-ef791bb4cbd8 service nova] Acquiring lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f7c956c4-e7b2-4557-badc-fd4e665bc8b9 req-bb17b551-fa48-499c-86b3-ef791bb4cbd8 service nova] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f7c956c4-e7b2-4557-badc-fd4e665bc8b9 req-bb17b551-fa48-499c-86b3-ef791bb4cbd8 service nova] Lock "4a1c695b-2ca4-4652-8fb6-19accdab6ebe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-f7c956c4-e7b2-4557-badc-fd4e665bc8b9 req-bb17b551-fa48-499c-86b3-ef791bb4cbd8 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] No waiting events found dispatching network-vif-plugged-260596e7-554d-4e86-b146-fe487c284c83 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:28:21 user nova-compute[71317]: WARNING nova.compute.manager [req-f7c956c4-e7b2-4557-badc-fd4e665bc8b9 req-bb17b551-fa48-499c-86b3-ef791bb4cbd8 service nova] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Received unexpected event network-vif-plugged-260596e7-554d-4e86-b146-fe487c284c83 for instance with vm_state deleted and task_state None.
Apr 14 04:28:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:26 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446491.851917, 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:28:26 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] VM Stopped (Lifecycle Event)
Apr 14 04:28:26 user nova-compute[71317]: DEBUG nova.compute.manager [None req-14e92bfd-7b53-47e7-a773-8e7606b80046 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:28:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:29 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:28:29 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:28:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:28:30 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:28:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:28:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:28:30 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:28:30 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Updating instance_info_cache with network_info: [{"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:28:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:28:30 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:28:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:28:31 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:28:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:28:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:28:35 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446500.0137906, 4a1c695b-2ca4-4652-8fb6-19accdab6ebe => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:28:35 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] VM Stopped (Lifecycle Event)
Apr 14 04:28:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-cc1460d3-18b6-4b83-8acb-f087f53d6cb2 None None] [instance: 4a1c695b-2ca4-4652-8fb6-19accdab6ebe] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:28:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:28:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:28:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:28:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:28:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:28:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.130s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:37 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.rescue --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk.rescue --force-share --output=json" returned: 0 in 0.130s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:39 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:28:39 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7747MB free_disk=26.25695037841797GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance f593ceb0-de44-490b-b365-3a05a3acd066 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 5273d202-ffc5-43c6-bb4f-f6d1a6855271 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance e19eb131-ae90-4220-9d47-15e8b8b332de actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:28:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.463s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:28:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:47 user nova-compute[71317]: INFO nova.compute.manager [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Terminating instance
Apr 14 04:28:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-4cf20fe3-749a-4c74-84eb-403500dff98a req-33a870a1-e948-4acb-8814-2184f2c542d8 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-vif-unplugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4cf20fe3-749a-4c74-84eb-403500dff98a req-33a870a1-e948-4acb-8814-2184f2c542d8 service nova] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4cf20fe3-749a-4c74-84eb-403500dff98a req-33a870a1-e948-4acb-8814-2184f2c542d8 service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4cf20fe3-749a-4c74-84eb-403500dff98a req-33a870a1-e948-4acb-8814-2184f2c542d8 service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-4cf20fe3-749a-4c74-84eb-403500dff98a req-33a870a1-e948-4acb-8814-2184f2c542d8 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] No waiting events found dispatching network-vif-unplugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:28:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-4cf20fe3-749a-4c74-84eb-403500dff98a req-33a870a1-e948-4acb-8814-2184f2c542d8 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-vif-unplugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-698bb1b9-f829-43f1-9dba-1f0d50c4903a req-96f3edc5-b458-4c77-8eed-426ba00c0da7 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Received event network-changed-fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-698bb1b9-f829-43f1-9dba-1f0d50c4903a req-96f3edc5-b458-4c77-8eed-426ba00c0da7 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Refreshing instance network info cache due to event network-changed-fab305ad-6649-4e13-944b-82a2dbc76547. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-698bb1b9-f829-43f1-9dba-1f0d50c4903a req-96f3edc5-b458-4c77-8eed-426ba00c0da7 service nova] Acquiring lock "refresh_cache-e19eb131-ae90-4220-9d47-15e8b8b332de" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-698bb1b9-f829-43f1-9dba-1f0d50c4903a req-96f3edc5-b458-4c77-8eed-426ba00c0da7 service nova] Acquired lock "refresh_cache-e19eb131-ae90-4220-9d47-15e8b8b332de" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.network.neutron [req-698bb1b9-f829-43f1-9dba-1f0d50c4903a req-96f3edc5-b458-4c77-8eed-426ba00c0da7 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Refreshing network info cache for port fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:28:48 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Instance destroyed successfully.
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.objects.instance [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lazy-loading 'resources' on Instance uuid 5273d202-ffc5-43c6-bb4f-f6d1a6855271 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:23:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-593822043',display_name='tempest-TestMinimumBasicScenario-server-593822043',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-593822043',id=23,image_ref='0b860afe-1945-4661-a55b-1a6ae0f644ea',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLyXBaFsreuBQDAM6pDqgr0rCR6Xh6YpqhuY/rJBXhVu6O6Jb6D2jMCp0EWMBknFKHHS7zo9OjEuZQRy7TrI+gfDrzjImHlLab3mcGep976OxV4nuHSlwuq9G12jktVdmA==',key_name='tempest-TestMinimumBasicScenario-231821443',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:23:58Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0a60f60717b44f4ba904a949eabfe608',ramdisk_id='',reservation_id='r-sji3zlhc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='0b860afe-1945-4661-a55b-1a6ae0f644ea',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1872250892',owner_user_name='tempest-TestMinimumBasicScenario-1872250892-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:26:08Z,user_data=None,user_id='982f167333954612a51911fc8d6c1ce9',uuid=5273d202-ffc5-43c6-bb4f-f6d1a6855271,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converting VIF {"id": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "address": "fa:16:3e:27:5d:35", "network": {"id": "d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-60282582-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0a60f60717b44f4ba904a949eabfe608", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc662cd5-f7", "ovs_interfaceid": "dc662cd5-f74a-40ee-bc14-e0948cbc4f36", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:27:5d:35,bridge_name='br-int',has_traffic_filtering=True,id=dc662cd5-f74a-40ee-bc14-e0948cbc4f36,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc662cd5-f7') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG os_vif [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:27:5d:35,bridge_name='br-int',has_traffic_filtering=True,id=dc662cd5-f74a-40ee-bc14-e0948cbc4f36,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc662cd5-f7') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapdc662cd5-f7, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:28:48 user nova-compute[71317]: INFO os_vif [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:27:5d:35,bridge_name='br-int',has_traffic_filtering=True,id=dc662cd5-f74a-40ee-bc14-e0948cbc4f36,network=Network(d3a59e6b-aaf6-4acb-a1cb-29f980dfa5a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc662cd5-f7')
Apr 14 04:28:48 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Deleting instance files /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271_del
Apr 14 04:28:48 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Deletion of /opt/stack/data/nova/instances/5273d202-ffc5-43c6-bb4f-f6d1a6855271_del complete
Apr 14 04:28:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:48 user nova-compute[71317]: INFO nova.compute.manager [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Took 0.91 seconds to destroy the instance on the hypervisor.
Apr 14 04:28:48 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:28:48 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Took 0.56 seconds to deallocate network for instance.
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.network.neutron [req-698bb1b9-f829-43f1-9dba-1f0d50c4903a req-96f3edc5-b458-4c77-8eed-426ba00c0da7 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Updated VIF entry in instance network info cache for port fab305ad-6649-4e13-944b-82a2dbc76547. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.network.neutron [req-698bb1b9-f829-43f1-9dba-1f0d50c4903a req-96f3edc5-b458-4c77-8eed-426ba00c0da7 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Updating instance_info_cache with network_info: [{"id": "fab305ad-6649-4e13-944b-82a2dbc76547", "address": "fa:16:3e:ec:97:fb", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfab305ad-66", "ovs_interfaceid": "fab305ad-6649-4e13-944b-82a2dbc76547", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-a48c1f70-ac20-47df-b8c1-dc7b937fe539 req-185963de-cd6a-418a-ab21-626898f69653 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-vif-deleted-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:48 user nova-compute[71317]: INFO nova.compute.manager [req-a48c1f70-ac20-47df-b8c1-dc7b937fe539 req-185963de-cd6a-418a-ab21-626898f69653 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Neutron deleted interface dc662cd5-f74a-40ee-bc14-e0948cbc4f36; detaching it from the instance and deleting it from the info cache
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.network.neutron [req-a48c1f70-ac20-47df-b8c1-dc7b937fe539 req-185963de-cd6a-418a-ab21-626898f69653 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-698bb1b9-f829-43f1-9dba-1f0d50c4903a req-96f3edc5-b458-4c77-8eed-426ba00c0da7 service nova] Releasing lock "refresh_cache-e19eb131-ae90-4220-9d47-15e8b8b332de" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-a48c1f70-ac20-47df-b8c1-dc7b937fe539 req-185963de-cd6a-418a-ab21-626898f69653 service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Detach interface failed, port_id=dc662cd5-f74a-40ee-bc14-e0948cbc4f36, reason: Instance 5273d202-ffc5-43c6-bb4f-f6d1a6855271 could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:49 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:28:49 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:28:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.326s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:49 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Deleted allocations for instance 5273d202-ffc5-43c6-bb4f-f6d1a6855271
Apr 14 04:28:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e53c2d0-bf77-4f3a-a00d-a87d5f561594 tempest-TestMinimumBasicScenario-1872250892 tempest-TestMinimumBasicScenario-1872250892-project-member] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.990s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-03502063-9e20-4c49-918c-d9ea1b0d141e req-0dfc928c-7481-45b6-8324-67617fa7aa0c service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received event network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-03502063-9e20-4c49-918c-d9ea1b0d141e req-0dfc928c-7481-45b6-8324-67617fa7aa0c service nova] Acquiring lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-03502063-9e20-4c49-918c-d9ea1b0d141e req-0dfc928c-7481-45b6-8324-67617fa7aa0c service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-03502063-9e20-4c49-918c-d9ea1b0d141e req-0dfc928c-7481-45b6-8324-67617fa7aa0c service nova] Lock "5273d202-ffc5-43c6-bb4f-f6d1a6855271-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-03502063-9e20-4c49-918c-d9ea1b0d141e req-0dfc928c-7481-45b6-8324-67617fa7aa0c service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] No waiting events found dispatching network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:28:49 user nova-compute[71317]: WARNING nova.compute.manager [req-03502063-9e20-4c49-918c-d9ea1b0d141e req-0dfc928c-7481-45b6-8324-67617fa7aa0c service nova] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Received unexpected event network-vif-plugged-dc662cd5-f74a-40ee-bc14-e0948cbc4f36 for instance with vm_state deleted and task_state None.
Apr 14 04:28:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:50 user nova-compute[71317]: DEBUG nova.objects.instance [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid e19eb131-ae90-4220-9d47-15e8b8b332de {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:28:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.079s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:50 user nova-compute[71317]: INFO nova.compute.manager [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Attaching volume e11df2cb-b474-4faf-afdf-a1c4e7d905ad to /dev/vdb
Apr 14 04:28:50 user nova-compute[71317]: DEBUG os_brick.utils [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:28:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:50 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b594a010-1b84-429a-a87b-680bb8b7b525]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a14fea6-a658-4d31-af40-9a9e583bc151]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c67b4081-d8c7-4313-98a0-5a038a9d262d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e838f185-115e-4b01-94da-5deea3014c15]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:28:51 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:28:51 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[eed6d9cd-bf93-4b73-a8ad-bb2c712c6ebb]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG os_brick.utils [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] <== get_connector_properties: return (125ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:28:51 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Updating existing volume attachment record: 94406c52-cd69-4979-a2de-dea06283c6b6 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:28:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "af7befec-3f77-4eb7-be53-d1b1e791713e" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rETDPxvm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:28:55 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:28:55 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:28:55 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:28:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6503eabc-abc2-4ef5-b90b-757c5aa26ff9]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:28:55 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b384a765-f257-447b-9cb0-7932098a7677]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad] added
Apr 14 04:28:55 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[33c3aced-c925-46a3-a6ec-53698655f110]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5000acc1-80f7-4b21-a426-51caa731db1b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op update -n node.session.auth.username -v rETDPxvm {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:28:55 user nova-compute[71317]: INFO nova.compute.claims [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Claim successful on node user
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op update -n node.session.auth.username -v rETDPxvm" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4086b578-835e-4b55-8f94-c0ad5acaac37]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'rETDPxvm'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[375498dd-f523-411b-916c-17b76091a3c1]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f309c6bb-cd8c-4b3d-a0d5-4f2651b1420d]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:55 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:55 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:55 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:55 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:55 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:55 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --login" returned: 0 in 0.067s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a06f4ec9-2771-490c-be62-dcf37e46bfb2]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad, portal: 172.16.0.220,3260]
Apr 14 04:28:55 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad, portal: 172.16.0.220,3260] successful.
Apr 14 04:28:55 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[272a4f2e-b3ce-42f3-9d51-7a5489c4bc90]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[587a7de8-13f0-4beb-8c84-20b8588a1384]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:55 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:55 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:55 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:28:55 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:28:55 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:28:55 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:28:55 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:28:55 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.304s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] HCTL ('34', '-', '-', 0) found on session 37 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4348ed5a-68c1-42dd-ab17-c5a1ce428a20]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Searching for a device in session 37 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:28:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rETDPxvm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.423s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:28:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:28:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:28:56 user nova-compute[71317]: INFO nova.virt.block_device [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Booting with volume 300999d2-cadf-4299-8a05-85dec6b7d895 at /dev/vda
Apr 14 04:28:56 user nova-compute[71317]: DEBUG nova.policy [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bcf79de7da27495dbfd14c556b5b54c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '693d95858c624feaa60114aa89082186', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG os_brick.utils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[178332d0-8a4c-4d8b-9424-eb2fee919cac]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[59fb89cc-25f0-4b30-87c2-b5cf6b5ae5e3]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1c7fd2e-adec-46bb-96f2-4f36a814a668]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0699f7c-316a-4b78-a73b-04f9ff7207fd]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:28:56 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:28:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a248d306-d903-4b1f-99e5-e2506ae22a1b]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG os_brick.utils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== get_connector_properties: return (123ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Updating existing volume attachment record: 380712c0-d593-4894-9e7c-53a4bc13bbe7 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.338s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] <== connect_volume: return (1340ms) {'type': 'block', 'scsi_wwn': '23439393331376166', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23439393331376166', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG nova.objects.instance [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid e19eb131-ae90-4220-9d47-15e8b8b332de {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:28:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:28:56 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:28:56 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:28:56 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:28:56 user nova-compute[71317]:   <serial>e11df2cb-b474-4faf-afdf-a1c4e7d905ad</serial>
Apr 14 04:28:56 user nova-compute[71317]: </disk>
Apr 14 04:28:56 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:28:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:57 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Successfully created port: 0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:28:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:28:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:28:57 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No VIF found with MAC fa:16:3e:ec:97:fb, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:28:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9998eb89-ab73-4040-8321-89ceb17dde8e tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.622s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:57 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Successfully updated port: 0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:28:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "refresh_cache-af7befec-3f77-4eb7-be53-d1b1e791713e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:28:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquired lock "refresh_cache-af7befec-3f77-4eb7-be53-d1b1e791713e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:28:57 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e336311-5aff-46f3-a8e8-071581fb8cfd req-293b48a8-14a3-419f-bbbe-f31e55b056e4 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Received event network-changed-0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-5e336311-5aff-46f3-a8e8-071581fb8cfd req-293b48a8-14a3-419f-bbbe-f31e55b056e4 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Refreshing instance network info cache due to event network-changed-0b301be3-2fd0-4e29-9740-a2ce2193d477. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e336311-5aff-46f3-a8e8-071581fb8cfd req-293b48a8-14a3-419f-bbbe-f31e55b056e4 service nova] Acquiring lock "refresh_cache-af7befec-3f77-4eb7-be53-d1b1e791713e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Updating instance_info_cache with network_info: [{"id": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "address": "fa:16:3e:ff:82:75", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0b301be3-2f", "ovs_interfaceid": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Releasing lock "refresh_cache-af7befec-3f77-4eb7-be53-d1b1e791713e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Instance network_info: |[{"id": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "address": "fa:16:3e:ff:82:75", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0b301be3-2f", "ovs_interfaceid": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e336311-5aff-46f3-a8e8-071581fb8cfd req-293b48a8-14a3-419f-bbbe-f31e55b056e4 service nova] Acquired lock "refresh_cache-af7befec-3f77-4eb7-be53-d1b1e791713e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG nova.network.neutron [req-5e336311-5aff-46f3-a8e8-071581fb8cfd req-293b48a8-14a3-419f-bbbe-f31e55b056e4 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Refreshing network info cache for port 0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG nova.network.neutron [req-5e336311-5aff-46f3-a8e8-071581fb8cfd req-293b48a8-14a3-419f-bbbe-f31e55b056e4 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Updated VIF entry in instance network info cache for port 0b301be3-2fd0-4e29-9740-a2ce2193d477. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG nova.network.neutron [req-5e336311-5aff-46f3-a8e8-071581fb8cfd req-293b48a8-14a3-419f-bbbe-f31e55b056e4 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Updating instance_info_cache with network_info: [{"id": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "address": "fa:16:3e:ff:82:75", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0b301be3-2f", "ovs_interfaceid": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:28:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5e336311-5aff-46f3-a8e8-071581fb8cfd req-293b48a8-14a3-419f-bbbe-f31e55b056e4 service nova] Releasing lock "refresh_cache-af7befec-3f77-4eb7-be53-d1b1e791713e" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:28:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:59 user nova-compute[71317]: DEBUG nova.objects.instance [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'flavor' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:28:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.064s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:28:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:28:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:28:59 user nova-compute[71317]: INFO nova.compute.manager [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attaching volume 90c1922d-91ec-49ff-8993-980caa04a670 to /dev/vdb
Apr 14 04:28:59 user nova-compute[71317]: DEBUG os_brick.utils [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:28:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:28:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[95b3fe59-7045-47cd-bbe2-237b757bbe78]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[62557480-1002-44d7-b537-e6c23c1ffb87]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.035s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3aab7ff-cc62-4cee-abf5-4e56fd935797]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc572bd1-1380-4d38-ad9d-0c8b61ac287b]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:29:00 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:29:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[161a8571-8018-4b16-9565-ed62bf86df6c]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG os_brick.utils [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] <== get_connector_properties: return (308ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:29:00 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Updating existing volume attachment record: 31a1c42d-6a7c-46bc-8137-9fb1e07c987e {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.objects.instance [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid e19eb131-ae90-4220-9d47-15e8b8b332de {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.055s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:29:01 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Creating image(s)
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Ensure instance console log exists: /opt/stack/data/nova/instances/af7befec-3f77-4eb7-be53-d1b1e791713e/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Start _get_guest_xml network_info=[{"id": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "address": "fa:16:3e:ff:82:75", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0b301be3-2f", "ovs_interfaceid": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': True, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'myaLbCEo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'af7befec-3f77-4eb7-be53-d1b1e791713e', 'attached_at': '', 'detached_at': '', 'volume_id': '300999d2-cadf-4299-8a05-85dec6b7d895', 'serial': '300999d2-cadf-4299-8a05-85dec6b7d895'}, 'attachment_id': '380712c0-d593-4894-9e7c-53a4bc13bbe7', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:29:01 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:29:01 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'myaLbCEo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:29:01 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:29:01 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:29:01 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:01 user nova-compute[71317]: INFO nova.compute.manager [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Attaching volume 43b7dada-23d8-4db0-9be7-b0fff3287b15 to /dev/vdc
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260" returned: 21 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[46da1ab6-bcac-492a-8328-c3c41905513a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:29:01 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[78fd98cb-56e5-4638-9ea8-08a9ca32b7d0]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895] added
Apr 14 04:29:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[381135d5-6ca6-437f-acfb-02eae10e7258]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.utils [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[22a165b0-0bb8-40e0-8086-148d4080122b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v myaLbCEo {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad0d9ae9-68d1-4217-a2a0-0f35f0cb33af]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v myaLbCEo" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0291ba75-0b3d-4330-af6e-963df192859f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'myaLbCEo'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a01f10e8-9d40-43d5-9ba0-eff15633b560]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c7f8797-5a7f-40a7-9cfa-6f793836737c]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b28415e-892d-4c5b-8d1d-e1278923b8f3]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[97bdb6fb-f2c7-46b8-9e65-5698c008ed38]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f037903-2864-4c0a-9101-5e1ac6fd238c]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:29:01 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1866acb8-42e7-4e00-a97e-9d21c7c967be]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.utils [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Updating existing volume attachment record: 7977ed42-2611-4e93-8fef-709adc21b4c3 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --login" returned: 0 in 0.079s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e001143-8610-44f5-ac59-0eab3fa218e4]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895, portal: 172.16.0.220,3260]
Apr 14 04:29:01 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895, portal: 172.16.0.220,3260] successful.
Apr 14 04:29:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[65c5eca3-4167-4bf0-89d4-4a35d8a846f2]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b52e1c0b-5f7f-46fa-906b-ceb4d5c6ae45]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:01 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.353s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] HCTL ('36', '-', '-', 0) found on session 38 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[336cfbf7-06d3-4392-be0f-b8223d0e8c2e]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Searching for a device in session 38 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:29:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'myaLbCEo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.385s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== connect_volume: return (1387ms) {'type': 'block', 'scsi_wwn': '23931653435353066', 'path': '/dev/sde'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23931653435353066', 'path': '/dev/sde'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:28:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-2095931005',display_name='tempest-TestVolumeBootPattern-volume-backed-server-2095931005',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-2095931005',id=30,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-bma2a5v4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:28:56Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=af7befec-3f77-4eb7-be53-d1b1e791713e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "address": "fa:16:3e:ff:82:75", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0b301be3-2f", "ovs_interfaceid": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "address": "fa:16:3e:ff:82:75", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0b301be3-2f", "ovs_interfaceid": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ff:82:75,bridge_name='br-int',has_traffic_filtering=True,id=0b301be3-2fd0-4e29-9740-a2ce2193d477,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0b301be3-2f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG nova.objects.instance [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lazy-loading 'pci_devices' on Instance uuid af7befec-3f77-4eb7-be53-d1b1e791713e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:29:02 user nova-compute[71317]:   <uuid>af7befec-3f77-4eb7-be53-d1b1e791713e</uuid>
Apr 14 04:29:02 user nova-compute[71317]:   <name>instance-0000001e</name>
Apr 14 04:29:02 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:29:02 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:29:02 user nova-compute[71317]:   <metadata>
Apr 14 04:29:02 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:29:02 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:29:02 user nova-compute[71317]:       <nova:name>tempest-TestVolumeBootPattern-volume-backed-server-2095931005</nova:name>
Apr 14 04:29:02 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:29:01</nova:creationTime>
Apr 14 04:29:02 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:29:02 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:29:02 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:29:02 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:29:02 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:29:02 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:29:02 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:29:02 user nova-compute[71317]:       <nova:owner>
Apr 14 04:29:02 user nova-compute[71317]:         <nova:user uuid="bcf79de7da27495dbfd14c556b5b54c7">tempest-TestVolumeBootPattern-1780160917-project-member</nova:user>
Apr 14 04:29:02 user nova-compute[71317]:         <nova:project uuid="693d95858c624feaa60114aa89082186">tempest-TestVolumeBootPattern-1780160917</nova:project>
Apr 14 04:29:02 user nova-compute[71317]:       </nova:owner>
Apr 14 04:29:02 user nova-compute[71317]:       <nova:ports>
Apr 14 04:29:02 user nova-compute[71317]:         <nova:port uuid="0b301be3-2fd0-4e29-9740-a2ce2193d477">
Apr 14 04:29:02 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 14 04:29:02 user nova-compute[71317]:         </nova:port>
Apr 14 04:29:02 user nova-compute[71317]:       </nova:ports>
Apr 14 04:29:02 user nova-compute[71317]:     </nova:instance>
Apr 14 04:29:02 user nova-compute[71317]:   </metadata>
Apr 14 04:29:02 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:29:02 user nova-compute[71317]:     <system>
Apr 14 04:29:02 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:29:02 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:29:02 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:29:02 user nova-compute[71317]:       <entry name="serial">af7befec-3f77-4eb7-be53-d1b1e791713e</entry>
Apr 14 04:29:02 user nova-compute[71317]:       <entry name="uuid">af7befec-3f77-4eb7-be53-d1b1e791713e</entry>
Apr 14 04:29:02 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:29:02 user nova-compute[71317]:     </system>
Apr 14 04:29:02 user nova-compute[71317]:   </sysinfo>
Apr 14 04:29:02 user nova-compute[71317]:   <os>
Apr 14 04:29:02 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:29:02 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:29:02 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:29:02 user nova-compute[71317]:   </os>
Apr 14 04:29:02 user nova-compute[71317]:   <features>
Apr 14 04:29:02 user nova-compute[71317]:     <acpi/>
Apr 14 04:29:02 user nova-compute[71317]:     <apic/>
Apr 14 04:29:02 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:29:02 user nova-compute[71317]:   </features>
Apr 14 04:29:02 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:29:02 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:29:02 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:29:02 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:29:02 user nova-compute[71317]:   </clock>
Apr 14 04:29:02 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:29:02 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:29:02 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:29:02 user nova-compute[71317]:   </cpu>
Apr 14 04:29:02 user nova-compute[71317]:   <devices>
Apr 14 04:29:02 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:29:02 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:29:02 user nova-compute[71317]:       <source dev="/dev/sde"/>
Apr 14 04:29:02 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:29:02 user nova-compute[71317]:       <serial>300999d2-cadf-4299-8a05-85dec6b7d895</serial>
Apr 14 04:29:02 user nova-compute[71317]:     </disk>
Apr 14 04:29:02 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:29:02 user nova-compute[71317]:       <mac address="fa:16:3e:ff:82:75"/>
Apr 14 04:29:02 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:29:02 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:29:02 user nova-compute[71317]:       <target dev="tap0b301be3-2f"/>
Apr 14 04:29:02 user nova-compute[71317]:     </interface>
Apr 14 04:29:02 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:29:02 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/af7befec-3f77-4eb7-be53-d1b1e791713e/console.log" append="off"/>
Apr 14 04:29:02 user nova-compute[71317]:     </serial>
Apr 14 04:29:02 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:29:02 user nova-compute[71317]:     <video>
Apr 14 04:29:02 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:29:02 user nova-compute[71317]:     </video>
Apr 14 04:29:02 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:29:02 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:29:02 user nova-compute[71317]:     </rng>
Apr 14 04:29:02 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:29:02 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:29:02 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:29:02 user nova-compute[71317]:     </memballoon>
Apr 14 04:29:02 user nova-compute[71317]:   </devices>
Apr 14 04:29:02 user nova-compute[71317]: </domain>
Apr 14 04:29:02 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:28:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-2095931005',display_name='tempest-TestVolumeBootPattern-volume-backed-server-2095931005',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-2095931005',id=30,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-bma2a5v4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:28:56Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=af7befec-3f77-4eb7-be53-d1b1e791713e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "address": "fa:16:3e:ff:82:75", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0b301be3-2f", "ovs_interfaceid": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "address": "fa:16:3e:ff:82:75", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0b301be3-2f", "ovs_interfaceid": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ff:82:75,bridge_name='br-int',has_traffic_filtering=True,id=0b301be3-2fd0-4e29-9740-a2ce2193d477,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0b301be3-2f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG os_vif [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ff:82:75,bridge_name='br-int',has_traffic_filtering=True,id=0b301be3-2fd0-4e29-9740-a2ce2193d477,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0b301be3-2f') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap0b301be3-2f, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap0b301be3-2f, col_values=(('external_ids', {'iface-id': '0b301be3-2fd0-4e29-9740-a2ce2193d477', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ff:82:75', 'vm-uuid': 'af7befec-3f77-4eb7-be53-d1b1e791713e'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:02 user nova-compute[71317]: INFO os_vif [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ff:82:75,bridge_name='br-int',has_traffic_filtering=True,id=0b301be3-2fd0-4e29-9740-a2ce2193d477,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0b301be3-2f')
Apr 14 04:29:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:29:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] No VIF found with MAC fa:16:3e:ff:82:75, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:29:03 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446528.1381571, 5273d202-ffc5-43c6-bb4f-f6d1a6855271 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:29:03 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] VM Stopped (Lifecycle Event)
Apr 14 04:29:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-52260ec7-3f12-4932-bbf2-9fafe05e6858 None None] [instance: 5273d202-ffc5-43c6-bb4f-f6d1a6855271] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:29:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-1e44a58c-951d-46c3-931e-e5fd707bc2eb req-1bc36952-60e2-448e-84d2-de4ef433382f service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Received event network-vif-plugged-0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:29:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1e44a58c-951d-46c3-931e-e5fd707bc2eb req-1bc36952-60e2-448e-84d2-de4ef433382f service nova] Acquiring lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1e44a58c-951d-46c3-931e-e5fd707bc2eb req-1bc36952-60e2-448e-84d2-de4ef433382f service nova] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1e44a58c-951d-46c3-931e-e5fd707bc2eb req-1bc36952-60e2-448e-84d2-de4ef433382f service nova] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-1e44a58c-951d-46c3-931e-e5fd707bc2eb req-1bc36952-60e2-448e-84d2-de4ef433382f service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] No waiting events found dispatching network-vif-plugged-0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:29:04 user nova-compute[71317]: WARNING nova.compute.manager [req-1e44a58c-951d-46c3-931e-e5fd707bc2eb req-1bc36952-60e2-448e-84d2-de4ef433382f service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Received unexpected event network-vif-plugged-0b301be3-2fd0-4e29-9740-a2ce2193d477 for instance with vm_state building and task_state spawning.
Apr 14 04:29:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wAqINGHP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:29:05 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:29:05 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:29:05 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260" returned: 21 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f682669-a116-43d1-af1e-f61ffcbdc5ff]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:29:05 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd394f4a-717c-4418-8950-46ee65d9a651]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670] added
Apr 14 04:29:05 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1847d45-2a8d-4447-84ca-5324372adce7]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c822a61-cdec-446e-bf2f-183786af8a77]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wAqINGHP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wAqINGHP" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5adcb99-0ae3-4910-8bb1-40ed6a119961]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'wAqINGHP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1fadd46d-9eb1-4e32-affc-b2d49a7f8f89]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d18ddc52-5a44-4880-9beb-c583e86979aa]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --login" returned: 0 in 0.087s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa0f67bd-0e4c-4aad-a643-18f94368f998]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670, portal: 172.16.0.220,3260]
Apr 14 04:29:05 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670, portal: 172.16.0.220,3260] successful.
Apr 14 04:29:05 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd85361e-8c44-47f8-944a-82dcc4ad0fc4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d98d5de9-502f-43dd-bc9b-b18b4643bd7a]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 (non-flash)
Apr 14 04:29:05 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.326s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] HCTL ('37', '-', '-', 0) found on session 39 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[30dafcb8-0464-410f-988f-6d8addaa4f49]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Searching for a device in session 39 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:29:05 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wAqINGHP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BPQNykGu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:29:06 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446546.3385725, af7befec-3f77-4eb7-be53-d1b1e791713e => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:29:06 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] VM Resumed (Lifecycle Event)
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:29:06 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Instance spawned successfully.
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:29:06 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446546.3427515, af7befec-3f77-4eb7-be53-d1b1e791713e => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:29:06 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] VM Started (Lifecycle Event)
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.362s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] <== connect_volume: return (1364ms) {'type': 'block', 'scsi_wwn': '23865636636343639', 'path': '/dev/sdf'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23865636636343639', 'path': '/dev/sdf'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.252s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:29:06 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:29:06 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.objects.instance [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'flavor' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:29:06 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-88e83a50-998c-46ef-b949-fb067a773085 req-e0dd6915-308b-4ca9-b643-96abdecbe3df service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Received event network-vif-plugged-0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-88e83a50-998c-46ef-b949-fb067a773085 req-e0dd6915-308b-4ca9-b643-96abdecbe3df service nova] Acquiring lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-88e83a50-998c-46ef-b949-fb067a773085 req-e0dd6915-308b-4ca9-b643-96abdecbe3df service nova] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-88e83a50-998c-46ef-b949-fb067a773085 req-e0dd6915-308b-4ca9-b643-96abdecbe3df service nova] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-88e83a50-998c-46ef-b949-fb067a773085 req-e0dd6915-308b-4ca9-b643-96abdecbe3df service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] No waiting events found dispatching network-vif-plugged-0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:29:06 user nova-compute[71317]: WARNING nova.compute.manager [req-88e83a50-998c-46ef-b949-fb067a773085 req-e0dd6915-308b-4ca9-b643-96abdecbe3df service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Received unexpected event network-vif-plugged-0b301be3-2fd0-4e29-9740-a2ce2193d477 for instance with vm_state building and task_state spawning.
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260" returned: 21 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[045878c0-df55-4881-841b-4bc601d31c4d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:29:06 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:29:06 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:29:06 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:29:06 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:29:06 user nova-compute[71317]:   <serial>90c1922d-91ec-49ff-8993-980caa04a670</serial>
Apr 14 04:29:06 user nova-compute[71317]: </disk>
Apr 14 04:29:06 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc3034c3-cbd9-4e2d-a3ac-94aa8334d288]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15] added
Apr 14 04:29:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:06 user nova-compute[71317]: INFO nova.compute.manager [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Took 5.27 seconds to spawn the instance on the hypervisor.
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[52f3fd8c-357d-4590-87e3-49fe0841f53e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd320d6d-e568-4c70-ad6b-985898cda858]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v BPQNykGu {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v BPQNykGu" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0afa80a4-8071-457a-a7aa-4bd8a254b46a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'BPQNykGu'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:06 user nova-compute[71317]: INFO nova.compute.manager [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Took 10.99 seconds to build instance.
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9e47a9b-ad0f-4751-b0a1-65decb1f7f5e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-8dc52a68-e0d6-4c29-afcf-a93f323ccd72 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 11.078s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b976b06b-3f0d-4f4d-baf4-44acdf501a74]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --login" returned: 0 in 0.079s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[57a5a9cb-ac92-4f5c-8cfb-1a28c68b0c46]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15, portal: 172.16.0.220,3260]
Apr 14 04:29:06 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15, portal: 172.16.0.220,3260] successful.
Apr 14 04:29:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:06 user nova-compute[71317]: INFO nova.compute.manager [None req-58d4cc6f-3d16-4cfb-aee8-a1b54ad14449 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Unrescuing
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58d4cc6f-3d16-4cfb-aee8-a1b54ad14449 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58d4cc6f-3d16-4cfb-aee8-a1b54ad14449 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquired lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.network.neutron [None req-58d4cc6f-3d16-4cfb-aee8-a1b54ad14449 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[364a8634-aaa6-41c1-a208-aa5c6e8cc0f7]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.038s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcb787a7-198b-4163-8c23-bd33bede850d]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 (non-flash)
Apr 14 04:29:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.371s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] HCTL ('39', '-', '-', 0) found on session 40 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8a42c24-a3ca-40c8-8718-b21d2531e037]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Searching for a device in session 40 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Connected to sdh using {'target_iqn': 'iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BPQNykGu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:29:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] No VIF found with MAC fa:16:3e:17:59:61, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-04f34ad2-6881-4c9d-9898-a918ed540016 tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.203s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG nova.network.neutron [None req-58d4cc6f-3d16-4cfb-aee8-a1b54ad14449 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updating instance_info_cache with network_info: [{"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58d4cc6f-3d16-4cfb-aee8-a1b54ad14449 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Releasing lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG nova.objects.instance [None req-58d4cc6f-3d16-4cfb-aee8-a1b54ad14449 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'flavor' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-e0c411d5-755e-4355-b34c-cbb953bc91ee req-d958b08e-a724-4d07-a724-7c4e7c3a5bcb service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-unplugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e0c411d5-755e-4355-b34c-cbb953bc91ee req-d958b08e-a724-4d07-a724-7c4e7c3a5bcb service nova] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e0c411d5-755e-4355-b34c-cbb953bc91ee req-d958b08e-a724-4d07-a724-7c4e7c3a5bcb service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e0c411d5-755e-4355-b34c-cbb953bc91ee req-d958b08e-a724-4d07-a724-7c4e7c3a5bcb service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-e0c411d5-755e-4355-b34c-cbb953bc91ee req-d958b08e-a724-4d07-a724-7c4e7c3a5bcb service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] No waiting events found dispatching network-vif-unplugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:29:07 user nova-compute[71317]: WARNING nova.compute.manager [req-e0c411d5-755e-4355-b34c-cbb953bc91ee req-d958b08e-a724-4d07-a724-7c4e7c3a5bcb service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received unexpected event network-vif-unplugged-e3868733-d538-4469-9a3e-c02014ec9066 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:29:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.415s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] <== connect_volume: return (1667ms) {'type': 'block', 'scsi_wwn': '26531333435666438', 'path': '/dev/sdh'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26531333435666438', 'path': '/dev/sdh'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG nova.objects.instance [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid e19eb131-ae90-4220-9d47-15e8b8b332de {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:29:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:29:07 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:29:07 user nova-compute[71317]:   <source dev="/dev/sdh"/>
Apr 14 04:29:07 user nova-compute[71317]:   <target dev="vdc" bus="virtio"/>
Apr 14 04:29:07 user nova-compute[71317]:   <serial>43b7dada-23d8-4db0-9be7-b0fff3287b15</serial>
Apr 14 04:29:07 user nova-compute[71317]: </disk>
Apr 14 04:29:07 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:29:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:08 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Instance destroyed successfully.
Apr 14 04:29:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:29:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:29:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No BDM found with device name vdc, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:29:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] No VIF found with MAC fa:16:3e:ec:97:fb, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:29:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3b844cad-6161-4c3c-9e19-c0f4e265b84f tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.263s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:08 user nova-compute[71317]: INFO nova.compute.manager [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Detaching volume 90c1922d-91ec-49ff-8993-980caa04a670
Apr 14 04:29:08 user nova-compute[71317]: INFO nova.virt.block_device [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Attempting to driver detach volume 90c1922d-91ec-49ff-8993-980caa04a670 from mountpoint /dev/vdb
Apr 14 04:29:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Attempting to detach device vdb from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:29:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:29:08 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:29:08 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:29:08 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:29:08 user nova-compute[71317]:   <serial>90c1922d-91ec-49ff-8993-980caa04a670</serial>
Apr 14 04:29:08 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:29:08 user nova-compute[71317]: </disk>
Apr 14 04:29:08 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:29:08 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Successfully detached device vdb from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the persistent domain config.
Apr 14 04:29:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:29:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:29:08 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:29:08 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:29:08 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:29:08 user nova-compute[71317]:   <serial>90c1922d-91ec-49ff-8993-980caa04a670</serial>
Apr 14 04:29:08 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:29:08 user nova-compute[71317]: </disk>
Apr 14 04:29:08 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446549.3509493, f593ceb0-de44-490b-b365-3a05a3acd066 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:29:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Successfully detached device vdb from instance f593ceb0-de44-490b-b365-3a05a3acd066 from the live domain config.
Apr 14 04:29:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wAqINGHP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:29:09 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc983e2d-efbf-4d98-8523-d9d14469c852]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:29:09 user nova-compute[71317]: No targets found.
Apr 14 04:29:09 user nova-compute[71317]: iSNS:
Apr 14 04:29:09 user nova-compute[71317]: No targets found.
Apr 14 04:29:09 user nova-compute[71317]: STATIC:
Apr 14 04:29:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:29:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670
Apr 14 04:29:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15
Apr 14 04:29:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895
Apr 14 04:29:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad
Apr 14 04:29:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:29:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:29:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:09 user nova-compute[71317]: FIRMWARE:
Apr 14 04:29:09 user nova-compute[71317]: No targets found.
Apr 14 04:29:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:29:09 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:29:09 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5403b28-fd28-4cf4-ac69-927fc519953d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2441fa7d-6e72-4b9b-8d29-39294bc21bd7]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:09 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:09 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:09 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:09 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:09 user nova-compute[71317]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 (non-flash)
Apr 14 04:29:09 user nova-compute[71317]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 (non-flash)
Apr 14 04:29:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:09 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:09 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:09 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:09 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:09 user nova-compute[71317]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 (non-flash)
Apr 14 04:29:09 user nova-compute[71317]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 (non-flash)
Apr 14 04:29:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d180a170>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670'): ({'sdf'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Removing single pathed devices sdf {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bddde7e9-422b-4340-8a4a-4f260661ed9a]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9f1987f-bb6f-4b28-b489-baff486d421a]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Flushing IO for device /dev/sdf {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c05ce97f-23f5-4a00-9478-9d8c4a7cd25c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.057s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f5bb3b6-19b2-47eb-a2df-274cba831ad3]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.037s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[702b496d-3494-4ac3-83c6-a6a477d73c13]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:09 user nova-compute[71317]: INFO nova.compute.manager [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Detaching volume e11df2cb-b474-4faf-afdf-a1c4e7d905ad
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --logout" returned: 0 in 0.083s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a25cd321-34e6-4a37-94eb-767eec41b611]: (4, ('Logging out of session [sid: 39, target: iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670, portal: 172.16.0.220,3260]\nLogout of [sid: 39, target: iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 39, target: iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670, portal: 172.16.0.220,3260]
Apr 14 04:29:09 user nova-compute[71317]: Logout of [sid: 39, target: iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670, portal: 172.16.0.220,3260] successful.
Apr 14 04:29:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG nova.compute.manager [req-3a6b9369-e3cf-4bfc-b74f-fc72691fd8ce req-723f0f74-4669-4e5e-be73-662f6c26f784 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3a6b9369-e3cf-4bfc-b74f-fc72691fd8ce req-723f0f74-4669-4e5e-be73-662f6c26f784 service nova] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3a6b9369-e3cf-4bfc-b74f-fc72691fd8ce req-723f0f74-4669-4e5e-be73-662f6c26f784 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3a6b9369-e3cf-4bfc-b74f-fc72691fd8ce req-723f0f74-4669-4e5e-be73-662f6c26f784 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG nova.compute.manager [req-3a6b9369-e3cf-4bfc-b74f-fc72691fd8ce req-723f0f74-4669-4e5e-be73-662f6c26f784 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] No waiting events found dispatching network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:29:09 user nova-compute[71317]: WARNING nova.compute.manager [req-3a6b9369-e3cf-4bfc-b74f-fc72691fd8ce req-723f0f74-4669-4e5e-be73-662f6c26f784 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received unexpected event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:90c1922d-91ec-49ff-8993-980caa04a670 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.038s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b1fcd97-71e1-4bb8-8b03-7456c73bfb42]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.408s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] <== disconnect_volume: return (410ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:29:09 user nova-compute[71317]: INFO nova.virt.block_device [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Attempting to driver detach volume e11df2cb-b474-4faf-afdf-a1c4e7d905ad from mountpoint /dev/vdb
Apr 14 04:29:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Attempting to detach device vdb from instance e19eb131-ae90-4220-9d47-15e8b8b332de from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:29:09 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:29:09 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:29:09 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:29:09 user nova-compute[71317]:   <serial>e11df2cb-b474-4faf-afdf-a1c4e7d905ad</serial>
Apr 14 04:29:09 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:29:09 user nova-compute[71317]: </disk>
Apr 14 04:29:09 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:29:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully detached device vdb from instance e19eb131-ae90-4220-9d47-15e8b8b332de from the persistent domain config.
Apr 14 04:29:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance e19eb131-ae90-4220-9d47-15e8b8b332de from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:29:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:29:09 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:29:09 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:29:09 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:29:09 user nova-compute[71317]:   <serial>e11df2cb-b474-4faf-afdf-a1c4e7d905ad</serial>
Apr 14 04:29:09 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:29:09 user nova-compute[71317]: </disk>
Apr 14 04:29:09 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446550.4724467, e19eb131-ae90-4220-9d47-15e8b8b332de => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance e19eb131-ae90-4220-9d47-15e8b8b332de {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:29:10 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully detached device vdb from instance e19eb131-ae90-4220-9d47-15e8b8b332de from the live domain config.
Apr 14 04:29:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rETDPxvm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:29:10 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[59d8358e-6d20-4190-a169-ed7aa459fe48]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:29:10 user nova-compute[71317]: No targets found.
Apr 14 04:29:10 user nova-compute[71317]: iSNS:
Apr 14 04:29:10 user nova-compute[71317]: No targets found.
Apr 14 04:29:10 user nova-compute[71317]: STATIC:
Apr 14 04:29:10 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:29:10 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:10 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:10 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15
Apr 14 04:29:10 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:10 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:10 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895
Apr 14 04:29:10 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:10 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:10 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad
Apr 14 04:29:10 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:10 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:10 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:29:10 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:10 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:10 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:29:10 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:10 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:10 user nova-compute[71317]: FIRMWARE:
Apr 14 04:29:10 user nova-compute[71317]: No targets found.
Apr 14 04:29:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:29:10 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:29:10 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[092b3e1a-be39-4990-a7b3-0f4c3b00a8bf]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fed45329-8880-497e-a14b-0c340df9b2a0]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:10 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:10 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:10 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:10 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:10 user nova-compute[71317]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 (non-flash)
Apr 14 04:29:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:10 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:10 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:10 user nova-compute[71317]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad (non-flash)
Apr 14 04:29:10 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:10 user nova-compute[71317]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 (non-flash)
Apr 14 04:29:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8490532b00>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f20571e5-7b5e-4925-a3f9-5f81738be973]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a58da77d-7af4-448d-990a-916c399f0a28]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[79671fbe-bf4e-446f-bd15-2b86bd65fe5e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.043s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea890157-a79f-4790-af5e-7d71500736a7]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[933cc413-686d-4750-a55f-dc2b18fb7a45]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --logout" returned: 0 in 0.072s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8757e72-bc1e-4ff8-95d0-b76ec62db8bb]: (4, ('Logging out of session [sid: 37, target: iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad, portal: 172.16.0.220,3260]\nLogout of [sid: 37, target: iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 37, target: iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad, portal: 172.16.0.220,3260]
Apr 14 04:29:10 user nova-compute[71317]: Logout of [sid: 37, target: iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad, portal: 172.16.0.220,3260] successful.
Apr 14 04:29:10 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e11df2cb-b474-4faf-afdf-a1c4e7d905ad -p 172.16.0.220:3260 --op delete" returned: 0 in 0.037s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2aab1fd-df4a-40d9-a2cd-d300755f0029]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.363s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] <== disconnect_volume: return (366ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'flavor' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:29:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d07c6827-353d-479e-ab45-80aa4c20784d tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.347s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid e19eb131-ae90-4220-9d47-15e8b8b332de {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:29:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-b0cbb7c8-7e0d-44a1-8a54-9f66631c066c tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.050s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-b5d0dbfe-9694-4bca-bac6-90f489f1944b req-58784aaa-83d6-459b-b06f-aa17802ea4b2 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:29:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b5d0dbfe-9694-4bca-bac6-90f489f1944b req-58784aaa-83d6-459b-b06f-aa17802ea4b2 service nova] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b5d0dbfe-9694-4bca-bac6-90f489f1944b req-58784aaa-83d6-459b-b06f-aa17802ea4b2 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b5d0dbfe-9694-4bca-bac6-90f489f1944b req-58784aaa-83d6-459b-b06f-aa17802ea4b2 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-b5d0dbfe-9694-4bca-bac6-90f489f1944b req-58784aaa-83d6-459b-b06f-aa17802ea4b2 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] No waiting events found dispatching network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:29:11 user nova-compute[71317]: WARNING nova.compute.manager [req-b5d0dbfe-9694-4bca-bac6-90f489f1944b req-58784aaa-83d6-459b-b06f-aa17802ea4b2 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received unexpected event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:29:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-b5d0dbfe-9694-4bca-bac6-90f489f1944b req-58784aaa-83d6-459b-b06f-aa17802ea4b2 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:29:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b5d0dbfe-9694-4bca-bac6-90f489f1944b req-58784aaa-83d6-459b-b06f-aa17802ea4b2 service nova] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b5d0dbfe-9694-4bca-bac6-90f489f1944b req-58784aaa-83d6-459b-b06f-aa17802ea4b2 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b5d0dbfe-9694-4bca-bac6-90f489f1944b req-58784aaa-83d6-459b-b06f-aa17802ea4b2 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-b5d0dbfe-9694-4bca-bac6-90f489f1944b req-58784aaa-83d6-459b-b06f-aa17802ea4b2 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] No waiting events found dispatching network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:29:11 user nova-compute[71317]: WARNING nova.compute.manager [req-b5d0dbfe-9694-4bca-bac6-90f489f1944b req-58784aaa-83d6-459b-b06f-aa17802ea4b2 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received unexpected event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:12 user nova-compute[71317]: INFO nova.compute.manager [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Detaching volume 43b7dada-23d8-4db0-9be7-b0fff3287b15
Apr 14 04:29:12 user nova-compute[71317]: INFO nova.virt.block_device [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Attempting to driver detach volume 43b7dada-23d8-4db0-9be7-b0fff3287b15 from mountpoint /dev/vdc
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Attempting to detach device vdc from instance e19eb131-ae90-4220-9d47-15e8b8b332de from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:29:12 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:29:12 user nova-compute[71317]:   <source dev="/dev/sdh"/>
Apr 14 04:29:12 user nova-compute[71317]:   <target dev="vdc" bus="virtio"/>
Apr 14 04:29:12 user nova-compute[71317]:   <serial>43b7dada-23d8-4db0-9be7-b0fff3287b15</serial>
Apr 14 04:29:12 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Apr 14 04:29:12 user nova-compute[71317]: </disk>
Apr 14 04:29:12 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:29:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully detached device vdc from instance e19eb131-ae90-4220-9d47-15e8b8b332de from the persistent domain config.
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] (1/8): Attempting to detach device vdc with device alias virtio-disk2 from instance e19eb131-ae90-4220-9d47-15e8b8b332de from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:29:12 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:29:12 user nova-compute[71317]:   <source dev="/dev/sdh"/>
Apr 14 04:29:12 user nova-compute[71317]:   <target dev="vdc" bus="virtio"/>
Apr 14 04:29:12 user nova-compute[71317]:   <serial>43b7dada-23d8-4db0-9be7-b0fff3287b15</serial>
Apr 14 04:29:12 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Apr 14 04:29:12 user nova-compute[71317]: </disk>
Apr 14 04:29:12 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446552.6900206, e19eb131-ae90-4220-9d47-15e8b8b332de => virtio-disk2> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Start waiting for the detach event from libvirt for device vdc with device alias virtio-disk2 for instance e19eb131-ae90-4220-9d47-15e8b8b332de {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:29:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully detached device vdc from instance e19eb131-ae90-4220-9d47-15e8b8b332de from the live domain config.
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BPQNykGu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:29:12 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3855a533-2d4e-4221-91fc-498a821c411a]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:29:12 user nova-compute[71317]: No targets found.
Apr 14 04:29:12 user nova-compute[71317]: iSNS:
Apr 14 04:29:12 user nova-compute[71317]: No targets found.
Apr 14 04:29:12 user nova-compute[71317]: STATIC:
Apr 14 04:29:12 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:29:12 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:12 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:12 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15
Apr 14 04:29:12 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:12 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:12 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895
Apr 14 04:29:12 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:12 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:12 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:29:12 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:12 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:12 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:29:12 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:29:12 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:29:12 user nova-compute[71317]: FIRMWARE:
Apr 14 04:29:12 user nova-compute[71317]: No targets found.
Apr 14 04:29:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:29:12 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:29:12 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a126ce56-4d33-46c0-8c9c-260c0587ac77]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.045s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9c6c84b-424f-4dc2-9be7-45bfc2991ec8]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:12 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:12 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:12 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:12 user nova-compute[71317]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 (non-flash)
Apr 14 04:29:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:29:12 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:29:12 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:29:12 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:29:12 user nova-compute[71317]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 (non-flash)
Apr 14 04:29:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8490531900>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15'): ({'sdh'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Removing single pathed devices sdh {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[41d3251d-0cfc-4949-999d-956fb7839eb6]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[870fb0f9-c5b3-4816-a6fa-d4000bcc0ecb]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Flushing IO for device /dev/sdh {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd381564-5de6-4c88-b44b-388ca683a269]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for cab84e5c-99ba-4e61-bf68-0a02b31ded5c due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446552.86462, cab84e5c-99ba-4e61-bf68-0a02b31ded5c => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:29:12 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] VM Resumed (Lifecycle Event)
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-58d4cc6f-3d16-4cfb-aee8-a1b54ad14449 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:29:12 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446552.8656585, cab84e5c-99ba-4e61-bf68-0a02b31ded5c => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:29:12 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] VM Started (Lifecycle Event)
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.056s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ec186cd-aa99-4d74-8c10-7be4b4467c8d]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] SCSI volumes sdh have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c2b5257-510f-4c53-8f17-4fd482e86548]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --logout" returned: 0 in 0.070s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[767b1b98-f81e-445a-a62f-aba1c4cc31a3]: (4, ('Logging out of session [sid: 40, target: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15, portal: 172.16.0.220,3260]\nLogout of [sid: 40, target: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 40, target: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15, portal: 172.16.0.220,3260]
Apr 14 04:29:13 user nova-compute[71317]: Logout of [sid: 40, target: iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15, portal: 172.16.0.220,3260] successful.
Apr 14 04:29:13 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43b7dada-23d8-4db0-9be7-b0fff3287b15 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8cc16fb-2d7c-45ba-8fa8-e0917ada066d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:29:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:29:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.376s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:29:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] <== disconnect_volume: return (377ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:29:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:29:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:13 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'flavor' on Instance uuid e19eb131-ae90-4220-9d47-15e8b8b332de {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:29:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d1f6a631-7fc8-4a3a-91ec-5655b3495bbe tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.906s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:14 user nova-compute[71317]: INFO nova.compute.manager [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Terminating instance
Apr 14 04:29:14 user nova-compute[71317]: DEBUG nova.compute.manager [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:29:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-762d6596-3052-4a34-b64f-9635cad21da7 req-be0f9f70-d13f-4210-9a1e-df50356096e0 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Received event network-vif-unplugged-fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:29:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-762d6596-3052-4a34-b64f-9635cad21da7 req-be0f9f70-d13f-4210-9a1e-df50356096e0 service nova] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-762d6596-3052-4a34-b64f-9635cad21da7 req-be0f9f70-d13f-4210-9a1e-df50356096e0 service nova] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-762d6596-3052-4a34-b64f-9635cad21da7 req-be0f9f70-d13f-4210-9a1e-df50356096e0 service nova] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-762d6596-3052-4a34-b64f-9635cad21da7 req-be0f9f70-d13f-4210-9a1e-df50356096e0 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] No waiting events found dispatching network-vif-unplugged-fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:29:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-762d6596-3052-4a34-b64f-9635cad21da7 req-be0f9f70-d13f-4210-9a1e-df50356096e0 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Received event network-vif-unplugged-fab305ad-6649-4e13-944b-82a2dbc76547 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:29:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:16 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Instance destroyed successfully.
Apr 14 04:29:16 user nova-compute[71317]: DEBUG nova.objects.instance [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lazy-loading 'resources' on Instance uuid e19eb131-ae90-4220-9d47-15e8b8b332de {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:29:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:26:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1852573542',display_name='tempest-AttachVolumeTestJSON-server-1852573542',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1852573542',id=29,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPahbjwxSvxq0IxMz5/M3Sbn33mEC8YBDkFVpCOH/WAboRbjSbxsllitzmJB7SVS4TKY3MCm6q57KW3T5WFQQOYggXTJylYDmiHUt2/y+RsASQ2RPX3ud6TI2ErXR0uk4g==',key_name='tempest-keypair-1690869240',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:27:03Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6a89255339f74e55ad7c92967558bb81',ramdisk_id='',reservation_id='r-j2gf6jl3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-991906288',owner_user_name='tempest-AttachVolumeTestJSON-991906288-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:27:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7610b45003e34f3395268977612f6c94',uuid=e19eb131-ae90-4220-9d47-15e8b8b332de,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fab305ad-6649-4e13-944b-82a2dbc76547", "address": "fa:16:3e:ec:97:fb", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfab305ad-66", "ovs_interfaceid": "fab305ad-6649-4e13-944b-82a2dbc76547", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:29:16 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converting VIF {"id": "fab305ad-6649-4e13-944b-82a2dbc76547", "address": "fa:16:3e:ec:97:fb", "network": {"id": "f3cc5345-2e8a-458b-9670-1a780db6a37c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-717004461-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a89255339f74e55ad7c92967558bb81", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfab305ad-66", "ovs_interfaceid": "fab305ad-6649-4e13-944b-82a2dbc76547", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:29:16 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ec:97:fb,bridge_name='br-int',has_traffic_filtering=True,id=fab305ad-6649-4e13-944b-82a2dbc76547,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfab305ad-66') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:29:16 user nova-compute[71317]: DEBUG os_vif [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:ec:97:fb,bridge_name='br-int',has_traffic_filtering=True,id=fab305ad-6649-4e13-944b-82a2dbc76547,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfab305ad-66') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:29:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapfab305ad-66, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:29:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:29:16 user nova-compute[71317]: INFO os_vif [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:ec:97:fb,bridge_name='br-int',has_traffic_filtering=True,id=fab305ad-6649-4e13-944b-82a2dbc76547,network=Network(f3cc5345-2e8a-458b-9670-1a780db6a37c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfab305ad-66')
Apr 14 04:29:16 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Deleting instance files /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de_del
Apr 14 04:29:16 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Deletion of /opt/stack/data/nova/instances/e19eb131-ae90-4220-9d47-15e8b8b332de_del complete
Apr 14 04:29:16 user nova-compute[71317]: INFO nova.compute.manager [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Took 1.49 seconds to destroy the instance on the hypervisor.
Apr 14 04:29:16 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:29:16 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:29:16 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:29:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-9d066459-1efc-4aee-96fb-ed941406cca2 req-57d505d1-8b39-46f4-b6ba-800e5345c592 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Received event network-vif-plugged-fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:29:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9d066459-1efc-4aee-96fb-ed941406cca2 req-57d505d1-8b39-46f4-b6ba-800e5345c592 service nova] Acquiring lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9d066459-1efc-4aee-96fb-ed941406cca2 req-57d505d1-8b39-46f4-b6ba-800e5345c592 service nova] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9d066459-1efc-4aee-96fb-ed941406cca2 req-57d505d1-8b39-46f4-b6ba-800e5345c592 service nova] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-9d066459-1efc-4aee-96fb-ed941406cca2 req-57d505d1-8b39-46f4-b6ba-800e5345c592 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] No waiting events found dispatching network-vif-plugged-fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:29:17 user nova-compute[71317]: WARNING nova.compute.manager [req-9d066459-1efc-4aee-96fb-ed941406cca2 req-57d505d1-8b39-46f4-b6ba-800e5345c592 service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Received unexpected event network-vif-plugged-fab305ad-6649-4e13-944b-82a2dbc76547 for instance with vm_state active and task_state deleting.
Apr 14 04:29:17 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:29:17 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Took 1.38 seconds to deallocate network for instance.
Apr 14 04:29:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-281ecd1e-b4f0-4439-9a5d-b12af4b9a1bb req-a8a4c772-965a-436f-947d-c789fcc6caae service nova] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Received event network-vif-deleted-fab305ad-6649-4e13-944b-82a2dbc76547 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:29:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:18 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:29:18 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:29:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.324s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:18 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Deleted allocations for instance e19eb131-ae90-4220-9d47-15e8b8b332de
Apr 14 04:29:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-87c49308-34e4-47a7-9f62-d8042168a608 tempest-AttachVolumeTestJSON-991906288 tempest-AttachVolumeTestJSON-991906288-project-member] Lock "e19eb131-ae90-4220-9d47-15e8b8b332de" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.439s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:31 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446556.212233, e19eb131-ae90-4220-9d47-15e8b8b332de => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:29:31 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] VM Stopped (Lifecycle Event)
Apr 14 04:29:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c73e96b9-d457-4c50-9652-4b86b05d3483 None None] [instance: e19eb131-ae90-4220-9d47-15e8b8b332de] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:29:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:29:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Updating instance_info_cache with network_info: [{"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:29:33 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:29:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:29:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:37 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdg (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:29:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:29:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:40 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:29:40 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:29:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7765MB free_disk=26.332439422607422GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:29:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance f593ceb0-de44-490b-b365-3a05a3acd066 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance a0a20aeb-ed06-4c65-b87d-74f8f825457d actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance af7befec-3f77-4eb7-be53-d1b1e791713e actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 8 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1536MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=8 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.405s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:29:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:49 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:29:49 user nova-compute[71317]: INFO nova.compute.claims [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Claim successful on node user
Apr 14 04:29:50 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.384s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG nova.network.neutron [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:29:50 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:29:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG nova.policy [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '6ec7103e06e643a6b9b2c908ba8d7b06', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd5995b934b0546c0983387e46e3b3df4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:29:50 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Creating image(s)
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "/opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "/opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "/opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk 1073741824" returned: 0 in 0.085s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.237s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Checking if we can resize image /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Cannot resize image /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG nova.objects.instance [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'migration_context' on Instance uuid 19a166de-f0c9-43f2-8c88-585e0d5d22cc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Ensure instance console log exists: /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG nova.network.neutron [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Successfully created port: e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG nova.network.neutron [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Successfully updated port: e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "refresh_cache-19a166de-f0c9-43f2-8c88-585e0d5d22cc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquired lock "refresh_cache-19a166de-f0c9-43f2-8c88-585e0d5d22cc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:29:51 user nova-compute[71317]: DEBUG nova.network.neutron [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-6fda2180-c38e-43b4-ba75-5fe051c872b0 req-ea4e26f6-7c55-465c-97fa-d504e1bc9467 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Received event network-changed-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-6fda2180-c38e-43b4-ba75-5fe051c872b0 req-ea4e26f6-7c55-465c-97fa-d504e1bc9467 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Refreshing instance network info cache due to event network-changed-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6fda2180-c38e-43b4-ba75-5fe051c872b0 req-ea4e26f6-7c55-465c-97fa-d504e1bc9467 service nova] Acquiring lock "refresh_cache-19a166de-f0c9-43f2-8c88-585e0d5d22cc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Updating instance_info_cache with network_info: [{"id": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "address": "fa:16:3e:b6:df:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape4576e1c-df", "ovs_interfaceid": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Releasing lock "refresh_cache-19a166de-f0c9-43f2-8c88-585e0d5d22cc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Instance network_info: |[{"id": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "address": "fa:16:3e:b6:df:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape4576e1c-df", "ovs_interfaceid": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6fda2180-c38e-43b4-ba75-5fe051c872b0 req-ea4e26f6-7c55-465c-97fa-d504e1bc9467 service nova] Acquired lock "refresh_cache-19a166de-f0c9-43f2-8c88-585e0d5d22cc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.network.neutron [req-6fda2180-c38e-43b4-ba75-5fe051c872b0 req-ea4e26f6-7c55-465c-97fa-d504e1bc9467 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Refreshing network info cache for port e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Start _get_guest_xml network_info=[{"id": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "address": "fa:16:3e:b6:df:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape4576e1c-df", "ovs_interfaceid": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:29:52 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:29:52 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:29:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-717082241',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-717082241',id=31,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-tuh7ljtq',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:29:50Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=19a166de-f0c9-43f2-8c88-585e0d5d22cc,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "address": "fa:16:3e:b6:df:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape4576e1c-df", "ovs_interfaceid": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "address": "fa:16:3e:b6:df:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape4576e1c-df", "ovs_interfaceid": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b6:df:7e,bridge_name='br-int',has_traffic_filtering=True,id=e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape4576e1c-df') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.objects.instance [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'pci_devices' on Instance uuid 19a166de-f0c9-43f2-8c88-585e0d5d22cc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:29:52 user nova-compute[71317]:   <uuid>19a166de-f0c9-43f2-8c88-585e0d5d22cc</uuid>
Apr 14 04:29:52 user nova-compute[71317]:   <name>instance-0000001f</name>
Apr 14 04:29:52 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:29:52 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:29:52 user nova-compute[71317]:   <metadata>
Apr 14 04:29:52 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:29:52 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:29:52 user nova-compute[71317]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-717082241</nova:name>
Apr 14 04:29:52 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:29:52</nova:creationTime>
Apr 14 04:29:52 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:29:52 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:29:52 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:29:52 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:29:52 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:29:52 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:29:52 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:29:52 user nova-compute[71317]:       <nova:owner>
Apr 14 04:29:52 user nova-compute[71317]:         <nova:user uuid="6ec7103e06e643a6b9b2c908ba8d7b06">tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member</nova:user>
Apr 14 04:29:52 user nova-compute[71317]:         <nova:project uuid="d5995b934b0546c0983387e46e3b3df4">tempest-ServerBootFromVolumeStableRescueTest-247821842</nova:project>
Apr 14 04:29:52 user nova-compute[71317]:       </nova:owner>
Apr 14 04:29:52 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:29:52 user nova-compute[71317]:       <nova:ports>
Apr 14 04:29:52 user nova-compute[71317]:         <nova:port uuid="e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d">
Apr 14 04:29:52 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 14 04:29:52 user nova-compute[71317]:         </nova:port>
Apr 14 04:29:52 user nova-compute[71317]:       </nova:ports>
Apr 14 04:29:52 user nova-compute[71317]:     </nova:instance>
Apr 14 04:29:52 user nova-compute[71317]:   </metadata>
Apr 14 04:29:52 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:29:52 user nova-compute[71317]:     <system>
Apr 14 04:29:52 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:29:52 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:29:52 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:29:52 user nova-compute[71317]:       <entry name="serial">19a166de-f0c9-43f2-8c88-585e0d5d22cc</entry>
Apr 14 04:29:52 user nova-compute[71317]:       <entry name="uuid">19a166de-f0c9-43f2-8c88-585e0d5d22cc</entry>
Apr 14 04:29:52 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:29:52 user nova-compute[71317]:     </system>
Apr 14 04:29:52 user nova-compute[71317]:   </sysinfo>
Apr 14 04:29:52 user nova-compute[71317]:   <os>
Apr 14 04:29:52 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:29:52 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:29:52 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:29:52 user nova-compute[71317]:   </os>
Apr 14 04:29:52 user nova-compute[71317]:   <features>
Apr 14 04:29:52 user nova-compute[71317]:     <acpi/>
Apr 14 04:29:52 user nova-compute[71317]:     <apic/>
Apr 14 04:29:52 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:29:52 user nova-compute[71317]:   </features>
Apr 14 04:29:52 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:29:52 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:29:52 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:29:52 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:29:52 user nova-compute[71317]:   </clock>
Apr 14 04:29:52 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:29:52 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:29:52 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:29:52 user nova-compute[71317]:   </cpu>
Apr 14 04:29:52 user nova-compute[71317]:   <devices>
Apr 14 04:29:52 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:29:52 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:29:52 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk"/>
Apr 14 04:29:52 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:29:52 user nova-compute[71317]:     </disk>
Apr 14 04:29:52 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:29:52 user nova-compute[71317]:       <mac address="fa:16:3e:b6:df:7e"/>
Apr 14 04:29:52 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:29:52 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:29:52 user nova-compute[71317]:       <target dev="tape4576e1c-df"/>
Apr 14 04:29:52 user nova-compute[71317]:     </interface>
Apr 14 04:29:52 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:29:52 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/console.log" append="off"/>
Apr 14 04:29:52 user nova-compute[71317]:     </serial>
Apr 14 04:29:52 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:29:52 user nova-compute[71317]:     <video>
Apr 14 04:29:52 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:29:52 user nova-compute[71317]:     </video>
Apr 14 04:29:52 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:29:52 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:29:52 user nova-compute[71317]:     </rng>
Apr 14 04:29:52 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:29:52 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:29:52 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:29:52 user nova-compute[71317]:     </memballoon>
Apr 14 04:29:52 user nova-compute[71317]:   </devices>
Apr 14 04:29:52 user nova-compute[71317]: </domain>
Apr 14 04:29:52 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:29:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-717082241',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-717082241',id=31,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-tuh7ljtq',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:29:50Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=19a166de-f0c9-43f2-8c88-585e0d5d22cc,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "address": "fa:16:3e:b6:df:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape4576e1c-df", "ovs_interfaceid": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "address": "fa:16:3e:b6:df:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape4576e1c-df", "ovs_interfaceid": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b6:df:7e,bridge_name='br-int',has_traffic_filtering=True,id=e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape4576e1c-df') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG os_vif [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b6:df:7e,bridge_name='br-int',has_traffic_filtering=True,id=e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape4576e1c-df') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape4576e1c-df, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape4576e1c-df, col_values=(('external_ids', {'iface-id': 'e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b6:df:7e', 'vm-uuid': '19a166de-f0c9-43f2-8c88-585e0d5d22cc'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:52 user nova-compute[71317]: INFO os_vif [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b6:df:7e,bridge_name='br-int',has_traffic_filtering=True,id=e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape4576e1c-df')
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No VIF found with MAC fa:16:3e:b6:df:7e, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.network.neutron [req-6fda2180-c38e-43b4-ba75-5fe051c872b0 req-ea4e26f6-7c55-465c-97fa-d504e1bc9467 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Updated VIF entry in instance network info cache for port e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG nova.network.neutron [req-6fda2180-c38e-43b4-ba75-5fe051c872b0 req-ea4e26f6-7c55-465c-97fa-d504e1bc9467 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Updating instance_info_cache with network_info: [{"id": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "address": "fa:16:3e:b6:df:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape4576e1c-df", "ovs_interfaceid": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:29:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6fda2180-c38e-43b4-ba75-5fe051c872b0 req-ea4e26f6-7c55-465c-97fa-d504e1bc9467 service nova] Releasing lock "refresh_cache-19a166de-f0c9-43f2-8c88-585e0d5d22cc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:29:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-c8f03e6b-6f12-44c3-9fa2-e08288bca17d req-8a538126-043b-4595-a544-06f54e77b69f service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Received event network-vif-plugged-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:29:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c8f03e6b-6f12-44c3-9fa2-e08288bca17d req-8a538126-043b-4595-a544-06f54e77b69f service nova] Acquiring lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c8f03e6b-6f12-44c3-9fa2-e08288bca17d req-8a538126-043b-4595-a544-06f54e77b69f service nova] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c8f03e6b-6f12-44c3-9fa2-e08288bca17d req-8a538126-043b-4595-a544-06f54e77b69f service nova] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-c8f03e6b-6f12-44c3-9fa2-e08288bca17d req-8a538126-043b-4595-a544-06f54e77b69f service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] No waiting events found dispatching network-vif-plugged-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:29:54 user nova-compute[71317]: WARNING nova.compute.manager [req-c8f03e6b-6f12-44c3-9fa2-e08288bca17d req-8a538126-043b-4595-a544-06f54e77b69f service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Received unexpected event network-vif-plugged-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d for instance with vm_state building and task_state spawning.
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446596.025958, 19a166de-f0c9-43f2-8c88-585e0d5d22cc => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:29:56 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] VM Resumed (Lifecycle Event)
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:29:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Instance spawned successfully.
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:29:56 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446596.030285, 19a166de-f0c9-43f2-8c88-585e0d5d22cc => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:29:56 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] VM Started (Lifecycle Event)
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:29:56 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:29:56 user nova-compute[71317]: INFO nova.compute.manager [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Took 5.71 seconds to spawn the instance on the hypervisor.
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-0cb7f905-a060-417b-a70c-9b80cd1276d9 req-189b1b7d-8ce5-49dd-8351-b4fb684f23c2 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Received event network-vif-plugged-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0cb7f905-a060-417b-a70c-9b80cd1276d9 req-189b1b7d-8ce5-49dd-8351-b4fb684f23c2 service nova] Acquiring lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0cb7f905-a060-417b-a70c-9b80cd1276d9 req-189b1b7d-8ce5-49dd-8351-b4fb684f23c2 service nova] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0cb7f905-a060-417b-a70c-9b80cd1276d9 req-189b1b7d-8ce5-49dd-8351-b4fb684f23c2 service nova] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-0cb7f905-a060-417b-a70c-9b80cd1276d9 req-189b1b7d-8ce5-49dd-8351-b4fb684f23c2 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] No waiting events found dispatching network-vif-plugged-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:29:56 user nova-compute[71317]: WARNING nova.compute.manager [req-0cb7f905-a060-417b-a70c-9b80cd1276d9 req-189b1b7d-8ce5-49dd-8351-b4fb684f23c2 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Received unexpected event network-vif-plugged-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d for instance with vm_state building and task_state spawning.
Apr 14 04:29:56 user nova-compute[71317]: INFO nova.compute.manager [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Took 6.44 seconds to build instance.
Apr 14 04:29:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3e679a9b-52e2-4f9a-ad02-5d079d685cd9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.576s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:29:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:29:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:03 user nova-compute[71317]: INFO nova.compute.manager [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Unshelving
Apr 14 04:30:03 user nova-compute[71317]: INFO nova.virt.block_device [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Booting with volume bcb3b184-a10d-482d-8c49-1bee42873be4 at /dev/vda
Apr 14 04:30:03 user nova-compute[71317]: DEBUG os_brick.utils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[af8c07a0-ca32-4bd4-85d5-dad87ac6d2a8]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4aa9b883-c036-4f3a-a530-c11fd397a01f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa92b837-f5c3-4ed5-8cc1-7d2abd86591b]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e671a807-cd8d-480d-93c2-5d46b7bda13a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:30:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:30:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:30:04 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:30:04 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b36237d1-8498-4af8-b5ec-878936c113b1]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:30:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:30:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:30:04 user nova-compute[71317]: DEBUG os_brick.utils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] <== get_connector_properties: return (127ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:30:04 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updating existing volume attachment record: 481ca92d-3524-46cf-9ee5-774a508c88e2 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:30:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lazy-loading 'pci_requests' on Instance uuid 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:30:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lazy-loading 'numa_topology' on Instance uuid 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:30:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:30:11 user nova-compute[71317]: INFO nova.compute.claims [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Claim successful on node user
Apr 14 04:30:11 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:30:11 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:30:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.508s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:11 user nova-compute[71317]: INFO nova.network.neutron [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updating port 61ec63cd-2733-40c4-a8c6-9243a6a3c144 with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Apr 14 04:30:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquired lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.network.neutron [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-4bea416e-2765-449e-ba96-418c1fd35b22 req-18b17bd9-c785-40d8-bcea-57e8cb7600c2 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-changed-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-4bea416e-2765-449e-ba96-418c1fd35b22 req-18b17bd9-c785-40d8-bcea-57e8cb7600c2 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Refreshing instance network info cache due to event network-changed-61ec63cd-2733-40c4-a8c6-9243a6a3c144. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4bea416e-2765-449e-ba96-418c1fd35b22 req-18b17bd9-c785-40d8-bcea-57e8cb7600c2 service nova] Acquiring lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.network.neutron [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updating instance_info_cache with network_info: [{"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Releasing lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:30:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Creating image(s)
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Ensure instance console log exists: /opt/stack/data/nova/instances/6a510dd3-bb77-4f6d-bdfd-180ce1d31c71/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Start _get_guest_xml network_info=[{"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': True, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pRhHtNJB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '6a510dd3-bb77-4f6d-bdfd-180ce1d31c71', 'attached_at': '', 'detached_at': '', 'volume_id': 'bcb3b184-a10d-482d-8c49-1bee42873be4', 'serial': 'bcb3b184-a10d-482d-8c49-1bee42873be4'}, 'attachment_id': '481ca92d-3524-46cf-9ee5-774a508c88e2', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4bea416e-2765-449e-ba96-418c1fd35b22 req-18b17bd9-c785-40d8-bcea-57e8cb7600c2 service nova] Acquired lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.network.neutron [req-4bea416e-2765-449e-ba96-418c1fd35b22 req-18b17bd9-c785-40d8-bcea-57e8cb7600c2 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Refreshing network info cache for port 61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:30:13 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:30:13 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lazy-loading 'vcpu_model' on Instance uuid 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pRhHtNJB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:30:13 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:30:13 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:30:13 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260" returned: 21 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0d16a92-5d61-4b60-b5bc-a48278030c59]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:30:13 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd2ab020-72b6-4f25-b332-77bb487d3408]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4] added
Apr 14 04:30:13 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[641c88c9-2ea4-4d86-859d-ab73c5c0d81b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[11d5dcb3-0329-4eb2-9ac5-99cacc08c8bc]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pRhHtNJB {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pRhHtNJB" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b948cb0-dc44-40dd-9f5c-79b30220a933]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'pRhHtNJB'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c12cb447-204f-43d5-8ab6-154840455fc2]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2644b9d4-d109-4c2a-a481-0080352a77ac]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:30:13 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:30:13 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:30:13 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:30:13 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:30:13 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:30:13 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:30:13 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:30:13 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --login" returned: 0 in 0.073s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbc11b3b-d17f-48de-9aec-c08721adc4c1]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260]
Apr 14 04:30:14 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260] successful.
Apr 14 04:30:14 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccc955b8-5959-4278-84de-cd46ec68f638]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a8f7eda-b2b8-41d9-946b-507aa2885d13]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:30:14 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:30:14 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:30:14 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:30:14 user nova-compute[71317]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:30:14 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:30:14 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:30:14 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:30:14 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:30:14 user nova-compute[71317]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:30:14 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.352s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] HCTL ('34', '-', '-', 0) found on session 41 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:30:14 user nova-compute[71317]: INFO nova.compute.claims [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Claim successful on node user
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b014d38-724a-4fa1-8b9e-c218775f9aa6]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Searching for a device in session 41 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pRhHtNJB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.network.neutron [req-4bea416e-2765-449e-ba96-418c1fd35b22 req-18b17bd9-c785-40d8-bcea-57e8cb7600c2 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updated VIF entry in instance network info cache for port 61ec63cd-2733-40c4-a8c6-9243a6a3c144. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.network.neutron [req-4bea416e-2765-449e-ba96-418c1fd35b22 req-18b17bd9-c785-40d8-bcea-57e8cb7600c2 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updating instance_info_cache with network_info: [{"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4bea416e-2765-449e-ba96-418c1fd35b22 req-18b17bd9-c785-40d8-bcea-57e8cb7600c2 service nova] Releasing lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.532s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.network.neutron [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:30:14 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.policy [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ebe3f41b95824cd78193d0bbe206a66b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e6175a4372245a4a57dca1f12f7f007', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:30:14 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Creating image(s)
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "/opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "/opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "/opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.157s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.403s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] <== connect_volume: return (1407ms) {'type': 'block', 'scsi_wwn': '23164336261353930', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23164336261353930', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:26:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-2065386753',display_name='tempest-TestShelveInstance-server-2065386753',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-2065386753',id=28,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-TestShelveInstance-2060150193',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:26:32Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='b0e15d3d3e0943c4b6bd54abf2de5fee',ramdisk_id='',reservation_id='r-osj4jtne',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1458140851',owner_user_name='tempest-TestShelveInstance-1458140851-project-member'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:30:04Z,user_data=None,user_id='36b6981092424de781e8403946f956d3',uuid=6a510dd3-bb77-4f6d-bdfd-180ce1d31c71,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Converting VIF {"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lazy-loading 'pci_devices' on Instance uuid 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:30:15 user nova-compute[71317]:   <uuid>6a510dd3-bb77-4f6d-bdfd-180ce1d31c71</uuid>
Apr 14 04:30:15 user nova-compute[71317]:   <name>instance-0000001c</name>
Apr 14 04:30:15 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:30:15 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:30:15 user nova-compute[71317]:   <metadata>
Apr 14 04:30:15 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:30:15 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:30:15 user nova-compute[71317]:       <nova:name>tempest-TestShelveInstance-server-2065386753</nova:name>
Apr 14 04:30:15 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:30:13</nova:creationTime>
Apr 14 04:30:15 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:30:15 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:30:15 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:30:15 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:30:15 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:30:15 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:30:15 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:30:15 user nova-compute[71317]:       <nova:owner>
Apr 14 04:30:15 user nova-compute[71317]:         <nova:user uuid="36b6981092424de781e8403946f956d3">tempest-TestShelveInstance-1458140851-project-member</nova:user>
Apr 14 04:30:15 user nova-compute[71317]:         <nova:project uuid="b0e15d3d3e0943c4b6bd54abf2de5fee">tempest-TestShelveInstance-1458140851</nova:project>
Apr 14 04:30:15 user nova-compute[71317]:       </nova:owner>
Apr 14 04:30:15 user nova-compute[71317]:       <nova:ports>
Apr 14 04:30:15 user nova-compute[71317]:         <nova:port uuid="61ec63cd-2733-40c4-a8c6-9243a6a3c144">
Apr 14 04:30:15 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 14 04:30:15 user nova-compute[71317]:         </nova:port>
Apr 14 04:30:15 user nova-compute[71317]:       </nova:ports>
Apr 14 04:30:15 user nova-compute[71317]:     </nova:instance>
Apr 14 04:30:15 user nova-compute[71317]:   </metadata>
Apr 14 04:30:15 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:30:15 user nova-compute[71317]:     <system>
Apr 14 04:30:15 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:30:15 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:30:15 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:30:15 user nova-compute[71317]:       <entry name="serial">6a510dd3-bb77-4f6d-bdfd-180ce1d31c71</entry>
Apr 14 04:30:15 user nova-compute[71317]:       <entry name="uuid">6a510dd3-bb77-4f6d-bdfd-180ce1d31c71</entry>
Apr 14 04:30:15 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:30:15 user nova-compute[71317]:     </system>
Apr 14 04:30:15 user nova-compute[71317]:   </sysinfo>
Apr 14 04:30:15 user nova-compute[71317]:   <os>
Apr 14 04:30:15 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:30:15 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:30:15 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:30:15 user nova-compute[71317]:   </os>
Apr 14 04:30:15 user nova-compute[71317]:   <features>
Apr 14 04:30:15 user nova-compute[71317]:     <acpi/>
Apr 14 04:30:15 user nova-compute[71317]:     <apic/>
Apr 14 04:30:15 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:30:15 user nova-compute[71317]:   </features>
Apr 14 04:30:15 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:30:15 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:30:15 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:30:15 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:30:15 user nova-compute[71317]:   </clock>
Apr 14 04:30:15 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:30:15 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:30:15 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:30:15 user nova-compute[71317]:   </cpu>
Apr 14 04:30:15 user nova-compute[71317]:   <devices>
Apr 14 04:30:15 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:30:15 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:30:15 user nova-compute[71317]:       <source dev="/dev/sdc"/>
Apr 14 04:30:15 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:30:15 user nova-compute[71317]:       <serial>bcb3b184-a10d-482d-8c49-1bee42873be4</serial>
Apr 14 04:30:15 user nova-compute[71317]:     </disk>
Apr 14 04:30:15 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:30:15 user nova-compute[71317]:       <mac address="fa:16:3e:fb:2e:cd"/>
Apr 14 04:30:15 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:30:15 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:30:15 user nova-compute[71317]:       <target dev="tap61ec63cd-27"/>
Apr 14 04:30:15 user nova-compute[71317]:     </interface>
Apr 14 04:30:15 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:30:15 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/6a510dd3-bb77-4f6d-bdfd-180ce1d31c71/console.log" append="off"/>
Apr 14 04:30:15 user nova-compute[71317]:     </serial>
Apr 14 04:30:15 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:30:15 user nova-compute[71317]:     <video>
Apr 14 04:30:15 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:30:15 user nova-compute[71317]:     </video>
Apr 14 04:30:15 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:30:15 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:30:15 user nova-compute[71317]:     </rng>
Apr 14 04:30:15 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:30:15 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:30:15 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:30:15 user nova-compute[71317]:     </memballoon>
Apr 14 04:30:15 user nova-compute[71317]:   </devices>
Apr 14 04:30:15 user nova-compute[71317]: </domain>
Apr 14 04:30:15 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:26:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-2065386753',display_name='tempest-TestShelveInstance-server-2065386753',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-2065386753',id=28,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-TestShelveInstance-2060150193',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:26:32Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='b0e15d3d3e0943c4b6bd54abf2de5fee',ramdisk_id='',reservation_id='r-osj4jtne',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1458140851',owner_user_name='tempest-TestShelveInstance-1458140851-project-member'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:30:04Z,user_data=None,user_id='36b6981092424de781e8403946f956d3',uuid=6a510dd3-bb77-4f6d-bdfd-180ce1d31c71,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Converting VIF {"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG os_vif [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.173s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap61ec63cd-27, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap61ec63cd-27, col_values=(('external_ids', {'iface-id': '61ec63cd-2733-40c4-a8c6-9243a6a3c144', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:fb:2e:cd', 'vm-uuid': '6a510dd3-bb77-4f6d-bdfd-180ce1d31c71'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk 1073741824" returned: 0 in 0.090s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.267s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:15 user nova-compute[71317]: INFO os_vif [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27')
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.158s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Checking if we can resize image /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] No VIF found with MAC fa:16:3e:fb:2e:cd, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Cannot resize image /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.objects.instance [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lazy-loading 'migration_context' on Instance uuid 390ca845-5660-49d0-bd8a-fcfc39176ca5 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Ensure instance console log exists: /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:15 user nova-compute[71317]: DEBUG nova.network.neutron [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Successfully created port: 488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:30:16 user nova-compute[71317]: DEBUG nova.network.neutron [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Successfully updated port: 488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:30:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:30:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquired lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:30:16 user nova-compute[71317]: DEBUG nova.network.neutron [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:30:16 user nova-compute[71317]: DEBUG nova.compute.manager [req-28ff2545-2826-44db-b207-dbd2ef460127 req-c7e5d033-6c94-4864-9e51-8d6f77eba970 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Received event network-changed-488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:16 user nova-compute[71317]: DEBUG nova.compute.manager [req-28ff2545-2826-44db-b207-dbd2ef460127 req-c7e5d033-6c94-4864-9e51-8d6f77eba970 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Refreshing instance network info cache due to event network-changed-488d7017-4718-4554-80f0-7fb119efc417. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:30:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-28ff2545-2826-44db-b207-dbd2ef460127 req-c7e5d033-6c94-4864-9e51-8d6f77eba970 service nova] Acquiring lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:30:16 user nova-compute[71317]: DEBUG nova.network.neutron [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:30:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.network.neutron [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Updating instance_info_cache with network_info: [{"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Releasing lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Instance network_info: |[{"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-28ff2545-2826-44db-b207-dbd2ef460127 req-c7e5d033-6c94-4864-9e51-8d6f77eba970 service nova] Acquired lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.network.neutron [req-28ff2545-2826-44db-b207-dbd2ef460127 req-c7e5d033-6c94-4864-9e51-8d6f77eba970 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Refreshing network info cache for port 488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Start _get_guest_xml network_info=[{"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:30:17 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:30:17 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:30:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-530656870',display_name='tempest-TestStampPattern-server-530656870',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-530656870',id=32,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPy10nG4Vr3ssb6R59u94cqlTShzxPhKC2bNuw8xGLQYLxJf5JqesyHfE6AT1NUBBqZ5yv5gwPvV4khi3Sx9VWvIr6XsmH3ZmGby83ZqN/gypVHL9c9HqBsp1SNADUtbzA==',key_name='tempest-TestStampPattern-1877604753',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0e6175a4372245a4a57dca1f12f7f007',ramdisk_id='',reservation_id='r-xhxn1384',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestStampPattern-267011728',owner_user_name='tempest-TestStampPattern-267011728-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:30:15Z,user_data=None,user_id='ebe3f41b95824cd78193d0bbe206a66b',uuid=390ca845-5660-49d0-bd8a-fcfc39176ca5,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Converting VIF {"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:2f:d9:94,bridge_name='br-int',has_traffic_filtering=True,id=488d7017-4718-4554-80f0-7fb119efc417,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap488d7017-47') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lazy-loading 'pci_devices' on Instance uuid 390ca845-5660-49d0-bd8a-fcfc39176ca5 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:30:17 user nova-compute[71317]:   <uuid>390ca845-5660-49d0-bd8a-fcfc39176ca5</uuid>
Apr 14 04:30:17 user nova-compute[71317]:   <name>instance-00000020</name>
Apr 14 04:30:17 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:30:17 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:30:17 user nova-compute[71317]:   <metadata>
Apr 14 04:30:17 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:30:17 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:30:17 user nova-compute[71317]:       <nova:name>tempest-TestStampPattern-server-530656870</nova:name>
Apr 14 04:30:17 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:30:17</nova:creationTime>
Apr 14 04:30:17 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:30:17 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:30:17 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:30:17 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:30:17 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:30:17 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:30:17 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:30:17 user nova-compute[71317]:       <nova:owner>
Apr 14 04:30:17 user nova-compute[71317]:         <nova:user uuid="ebe3f41b95824cd78193d0bbe206a66b">tempest-TestStampPattern-267011728-project-member</nova:user>
Apr 14 04:30:17 user nova-compute[71317]:         <nova:project uuid="0e6175a4372245a4a57dca1f12f7f007">tempest-TestStampPattern-267011728</nova:project>
Apr 14 04:30:17 user nova-compute[71317]:       </nova:owner>
Apr 14 04:30:17 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:30:17 user nova-compute[71317]:       <nova:ports>
Apr 14 04:30:17 user nova-compute[71317]:         <nova:port uuid="488d7017-4718-4554-80f0-7fb119efc417">
Apr 14 04:30:17 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 14 04:30:17 user nova-compute[71317]:         </nova:port>
Apr 14 04:30:17 user nova-compute[71317]:       </nova:ports>
Apr 14 04:30:17 user nova-compute[71317]:     </nova:instance>
Apr 14 04:30:17 user nova-compute[71317]:   </metadata>
Apr 14 04:30:17 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:30:17 user nova-compute[71317]:     <system>
Apr 14 04:30:17 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:30:17 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:30:17 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:30:17 user nova-compute[71317]:       <entry name="serial">390ca845-5660-49d0-bd8a-fcfc39176ca5</entry>
Apr 14 04:30:17 user nova-compute[71317]:       <entry name="uuid">390ca845-5660-49d0-bd8a-fcfc39176ca5</entry>
Apr 14 04:30:17 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:30:17 user nova-compute[71317]:     </system>
Apr 14 04:30:17 user nova-compute[71317]:   </sysinfo>
Apr 14 04:30:17 user nova-compute[71317]:   <os>
Apr 14 04:30:17 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:30:17 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:30:17 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:30:17 user nova-compute[71317]:   </os>
Apr 14 04:30:17 user nova-compute[71317]:   <features>
Apr 14 04:30:17 user nova-compute[71317]:     <acpi/>
Apr 14 04:30:17 user nova-compute[71317]:     <apic/>
Apr 14 04:30:17 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:30:17 user nova-compute[71317]:   </features>
Apr 14 04:30:17 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:30:17 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:30:17 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:30:17 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:30:17 user nova-compute[71317]:   </clock>
Apr 14 04:30:17 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:30:17 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:30:17 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:30:17 user nova-compute[71317]:   </cpu>
Apr 14 04:30:17 user nova-compute[71317]:   <devices>
Apr 14 04:30:17 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:30:17 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:30:17 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk"/>
Apr 14 04:30:17 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:30:17 user nova-compute[71317]:     </disk>
Apr 14 04:30:17 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:30:17 user nova-compute[71317]:       <mac address="fa:16:3e:2f:d9:94"/>
Apr 14 04:30:17 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:30:17 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:30:17 user nova-compute[71317]:       <target dev="tap488d7017-47"/>
Apr 14 04:30:17 user nova-compute[71317]:     </interface>
Apr 14 04:30:17 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:30:17 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/console.log" append="off"/>
Apr 14 04:30:17 user nova-compute[71317]:     </serial>
Apr 14 04:30:17 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:30:17 user nova-compute[71317]:     <video>
Apr 14 04:30:17 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:30:17 user nova-compute[71317]:     </video>
Apr 14 04:30:17 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:30:17 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:30:17 user nova-compute[71317]:     </rng>
Apr 14 04:30:17 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:30:17 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:30:17 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:30:17 user nova-compute[71317]:     </memballoon>
Apr 14 04:30:17 user nova-compute[71317]:   </devices>
Apr 14 04:30:17 user nova-compute[71317]: </domain>
Apr 14 04:30:17 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:30:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-530656870',display_name='tempest-TestStampPattern-server-530656870',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-530656870',id=32,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPy10nG4Vr3ssb6R59u94cqlTShzxPhKC2bNuw8xGLQYLxJf5JqesyHfE6AT1NUBBqZ5yv5gwPvV4khi3Sx9VWvIr6XsmH3ZmGby83ZqN/gypVHL9c9HqBsp1SNADUtbzA==',key_name='tempest-TestStampPattern-1877604753',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0e6175a4372245a4a57dca1f12f7f007',ramdisk_id='',reservation_id='r-xhxn1384',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestStampPattern-267011728',owner_user_name='tempest-TestStampPattern-267011728-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:30:15Z,user_data=None,user_id='ebe3f41b95824cd78193d0bbe206a66b',uuid=390ca845-5660-49d0-bd8a-fcfc39176ca5,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Converting VIF {"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:2f:d9:94,bridge_name='br-int',has_traffic_filtering=True,id=488d7017-4718-4554-80f0-7fb119efc417,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap488d7017-47') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG os_vif [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:2f:d9:94,bridge_name='br-int',has_traffic_filtering=True,id=488d7017-4718-4554-80f0-7fb119efc417,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap488d7017-47') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap488d7017-47, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap488d7017-47, col_values=(('external_ids', {'iface-id': '488d7017-4718-4554-80f0-7fb119efc417', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:2f:d9:94', 'vm-uuid': '390ca845-5660-49d0-bd8a-fcfc39176ca5'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:17 user nova-compute[71317]: INFO os_vif [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:2f:d9:94,bridge_name='br-int',has_traffic_filtering=True,id=488d7017-4718-4554-80f0-7fb119efc417,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap488d7017-47')
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-b8e326c8-b850-41a6-b287-00d6aa537ba6 req-10a8926c-d145-4765-a617-4df8d85ffe98 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b8e326c8-b850-41a6-b287-00d6aa537ba6 req-10a8926c-d145-4765-a617-4df8d85ffe98 service nova] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b8e326c8-b850-41a6-b287-00d6aa537ba6 req-10a8926c-d145-4765-a617-4df8d85ffe98 service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b8e326c8-b850-41a6-b287-00d6aa537ba6 req-10a8926c-d145-4765-a617-4df8d85ffe98 service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-b8e326c8-b850-41a6-b287-00d6aa537ba6 req-10a8926c-d145-4765-a617-4df8d85ffe98 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] No waiting events found dispatching network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:30:17 user nova-compute[71317]: WARNING nova.compute.manager [req-b8e326c8-b850-41a6-b287-00d6aa537ba6 req-10a8926c-d145-4765-a617-4df8d85ffe98 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received unexpected event network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 for instance with vm_state shelved_offloaded and task_state spawning.
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] No VIF found with MAC fa:16:3e:2f:d9:94, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:17 user nova-compute[71317]: INFO nova.compute.manager [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Terminating instance
Apr 14 04:30:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a9ba507-3241-4423-9237-1c93993654e6 req-53917924-b1f8-4bcc-88cd-5e3c3ac8c27c service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-unplugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a9ba507-3241-4423-9237-1c93993654e6 req-53917924-b1f8-4bcc-88cd-5e3c3ac8c27c service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a9ba507-3241-4423-9237-1c93993654e6 req-53917924-b1f8-4bcc-88cd-5e3c3ac8c27c service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2a9ba507-3241-4423-9237-1c93993654e6 req-53917924-b1f8-4bcc-88cd-5e3c3ac8c27c service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a9ba507-3241-4423-9237-1c93993654e6 req-53917924-b1f8-4bcc-88cd-5e3c3ac8c27c service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-unplugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:30:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-2a9ba507-3241-4423-9237-1c93993654e6 req-53917924-b1f8-4bcc-88cd-5e3c3ac8c27c service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-unplugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:30:18 user nova-compute[71317]: DEBUG nova.network.neutron [req-28ff2545-2826-44db-b207-dbd2ef460127 req-c7e5d033-6c94-4864-9e51-8d6f77eba970 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Updated VIF entry in instance network info cache for port 488d7017-4718-4554-80f0-7fb119efc417. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:30:18 user nova-compute[71317]: DEBUG nova.network.neutron [req-28ff2545-2826-44db-b207-dbd2ef460127 req-c7e5d033-6c94-4864-9e51-8d6f77eba970 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Updating instance_info_cache with network_info: [{"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:30:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-28ff2545-2826-44db-b207-dbd2ef460127 req-c7e5d033-6c94-4864-9e51-8d6f77eba970 service nova] Releasing lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:30:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:19 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Instance destroyed successfully.
Apr 14 04:30:19 user nova-compute[71317]: DEBUG nova.objects.instance [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'resources' on Instance uuid a0a20aeb-ed06-4c65-b87d-74f8f825457d {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1350614351',display_name='tempest-ServerActionsTestJSON-server-1398117926',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1350614351',id=6,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:27:41Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-789bnjb3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:27:42Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=a0a20aeb-ed06-4c65-b87d-74f8f825457d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "address": "fa:16:3e:5d:8f:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3a3e3b13-7b", "ovs_interfaceid": "3a3e3b13-7b4e-46c8-9123-5c1f121a38b9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_vif [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3a3e3b13-7b, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-f7d2e9ad-a20f-4b49-b593-82c62a8ab68e req-41860fc3-a0e6-443b-801d-3a223e41cb47 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f7d2e9ad-a20f-4b49-b593-82c62a8ab68e req-41860fc3-a0e6-443b-801d-3a223e41cb47 service nova] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f7d2e9ad-a20f-4b49-b593-82c62a8ab68e req-41860fc3-a0e6-443b-801d-3a223e41cb47 service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f7d2e9ad-a20f-4b49-b593-82c62a8ab68e req-41860fc3-a0e6-443b-801d-3a223e41cb47 service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-f7d2e9ad-a20f-4b49-b593-82c62a8ab68e req-41860fc3-a0e6-443b-801d-3a223e41cb47 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] No waiting events found dispatching network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:30:19 user nova-compute[71317]: WARNING nova.compute.manager [req-f7d2e9ad-a20f-4b49-b593-82c62a8ab68e req-41860fc3-a0e6-443b-801d-3a223e41cb47 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received unexpected event network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 for instance with vm_state shelved_offloaded and task_state spawning.
Apr 14 04:30:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:19 user nova-compute[71317]: INFO os_vif [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5d:8f:83,bridge_name='br-int',has_traffic_filtering=True,id=3a3e3b13-7b4e-46c8-9123-5c1f121a38b9,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3a3e3b13-7b')
Apr 14 04:30:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MjAZDWGc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:30:19 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[762b83b3-ede3-4717-8b24-d323b99894a9]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:30:19 user nova-compute[71317]: No targets found.
Apr 14 04:30:19 user nova-compute[71317]: iSNS:
Apr 14 04:30:19 user nova-compute[71317]: No targets found.
Apr 14 04:30:19 user nova-compute[71317]: STATIC:
Apr 14 04:30:19 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:30:19 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:30:19 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:30:19 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895
Apr 14 04:30:19 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:30:19 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:30:19 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:30:19 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:30:19 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:30:19 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4
Apr 14 04:30:19 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:30:19 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:30:19 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2
Apr 14 04:30:19 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:30:19 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:30:19 user nova-compute[71317]: FIRMWARE:
Apr 14 04:30:19 user nova-compute[71317]: No targets found.
Apr 14 04:30:19 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:30:19 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:30:19 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[518a276b-6775-48ff-bcea-2d7c0568d79b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.037s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[47844f0e-f111-4a4f-bd57-d7c89051bf29]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:30:19 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:30:19 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:30:19 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:30:19 user nova-compute[71317]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:30:19 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:30:19 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:30:19 user nova-compute[71317]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 (non-flash)
Apr 14 04:30:19 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:30:19 user nova-compute[71317]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:30:19 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f849068cd30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2'): ({'sdd'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Removing single pathed devices sdd {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5068e82b-1ec1-416e-8a5c-095c926df9b2]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed3ba17e-6ce4-4cab-9263-628ec11fc8ba]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flushing IO for device /dev/sdd {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7df7a4d5-0393-48e4-950b-8a1e03b83a17]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.049s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d67915f-a002-4f27-80ec-468805199d6b]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[86e9c1eb-15f8-478b-9be7-49193ab64d9d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --logout" returned: 0 in 0.065s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3893f172-fefb-48ee-9245-1b60bfdd3d34]: (4, ('Logging out of session [sid: 34, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260]\nLogout of [sid: 34, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 34, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260]
Apr 14 04:30:19 user nova-compute[71317]: Logout of [sid: 34, target: iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2, portal: 172.16.0.220,3260] successful.
Apr 14 04:30:19 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9cdf2c58-c295-483e-b766-13e01129ede2 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.038s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[17cfc0be-c9df-44ef-bd72-97ed3c28ea87]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.399s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== disconnect_volume: return (402ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:30:19 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Deleting instance files /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d_del
Apr 14 04:30:19 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Deletion of /opt/stack/data/nova/instances/a0a20aeb-ed06-4c65-b87d-74f8f825457d_del complete
Apr 14 04:30:19 user nova-compute[71317]: INFO nova.compute.manager [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Took 2.23 seconds to destroy the instance on the hypervisor.
Apr 14 04:30:19 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:30:19 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] Acquiring lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] No waiting events found dispatching network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:30:20 user nova-compute[71317]: WARNING nova.compute.manager [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received unexpected event network-vif-plugged-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 for instance with vm_state active and task_state deleting.
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Received event network-vif-plugged-488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] Acquiring lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] No waiting events found dispatching network-vif-plugged-488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:30:20 user nova-compute[71317]: WARNING nova.compute.manager [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Received unexpected event network-vif-plugged-488d7017-4718-4554-80f0-7fb119efc417 for instance with vm_state building and task_state spawning.
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Received event network-vif-plugged-488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] Acquiring lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] No waiting events found dispatching network-vif-plugged-488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:30:20 user nova-compute[71317]: WARNING nova.compute.manager [req-1d505c86-80a9-48de-92b0-89aab7d553a0 req-479a42d4-1623-4f0e-a4af-79001696148e service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Received unexpected event network-vif-plugged-488d7017-4718-4554-80f0-7fb119efc417 for instance with vm_state building and task_state spawning.
Apr 14 04:30:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446620.8170202, 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:30:20 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] VM Resumed (Lifecycle Event)
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:30:20 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Instance spawned successfully.
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:30:20 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446620.8222113, 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:30:20 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] VM Started (Lifecycle Event)
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Synchronizing instance power state after lifecycle event "Started"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:30:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d7bc4828-10f8-429f-a487-6a5151cf59c1 tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 17.213s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:30:21 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Took 1.47 seconds to deallocate network for instance.
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-685f5623-6fbb-4ae0-b751-ae26e796fa31 req-282d6289-4552-4d96-83b4-daf019c77bd7 service nova] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Received event network-vif-deleted-3a3e3b13-7b4e-46c8-9123-5c1f121a38b9 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-806a2ff0-6a39-4b1f-8bd6-d381f1a8034d req-c49b38ba-db3a-475a-a279-899e5bc79700 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Received event network-changed-dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-806a2ff0-6a39-4b1f-8bd6-d381f1a8034d req-c49b38ba-db3a-475a-a279-899e5bc79700 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Refreshing instance network info cache due to event network-changed-dd12687e-ca27-4fba-9653-4fa1af3608a8. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-806a2ff0-6a39-4b1f-8bd6-d381f1a8034d req-c49b38ba-db3a-475a-a279-899e5bc79700 service nova] Acquiring lock "refresh_cache-f593ceb0-de44-490b-b365-3a05a3acd066" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-806a2ff0-6a39-4b1f-8bd6-d381f1a8034d req-c49b38ba-db3a-475a-a279-899e5bc79700 service nova] Acquired lock "refresh_cache-f593ceb0-de44-490b-b365-3a05a3acd066" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.network.neutron [req-806a2ff0-6a39-4b1f-8bd6-d381f1a8034d req-c49b38ba-db3a-475a-a279-899e5bc79700 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Refreshing network info cache for port dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446621.6617312, 390ca845-5660-49d0-bd8a-fcfc39176ca5 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:30:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] VM Resumed (Lifecycle Event)
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:30:21 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Instance spawned successfully.
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:30:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446621.6636992, 390ca845-5660-49d0-bd8a-fcfc39176ca5 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:30:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] VM Started (Lifecycle Event)
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:30:21 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:30:21 user nova-compute[71317]: INFO nova.compute.manager [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Took 6.93 seconds to spawn the instance on the hypervisor.
Apr 14 04:30:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:30:21 user nova-compute[71317]: INFO nova.compute.manager [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Took 7.85 seconds to build instance.
Apr 14 04:30:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4fd080a2-e60e-41fe-8298-1ddc66beeef1 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.976s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:22 user nova-compute[71317]: DEBUG nova.network.neutron [req-806a2ff0-6a39-4b1f-8bd6-d381f1a8034d req-c49b38ba-db3a-475a-a279-899e5bc79700 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Updated VIF entry in instance network info cache for port dd12687e-ca27-4fba-9653-4fa1af3608a8. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:30:22 user nova-compute[71317]: DEBUG nova.network.neutron [req-806a2ff0-6a39-4b1f-8bd6-d381f1a8034d req-c49b38ba-db3a-475a-a279-899e5bc79700 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Updating instance_info_cache with network_info: [{"id": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "address": "fa:16:3e:17:59:61", "network": {"id": "694b7c0a-cb94-4b83-99aa-03c94c049086", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-383837474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "39731dbb51484eff890a74b07241ba75", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd12687e-ca", "ovs_interfaceid": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:30:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-806a2ff0-6a39-4b1f-8bd6-d381f1a8034d req-c49b38ba-db3a-475a-a279-899e5bc79700 service nova] Releasing lock "refresh_cache-f593ceb0-de44-490b-b365-3a05a3acd066" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:30:22 user nova-compute[71317]: INFO nova.compute.manager [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Took 1.00 seconds to detach 1 volumes for instance.
Apr 14 04:30:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:22 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:30:22 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:30:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.484s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:23 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Deleted allocations for instance a0a20aeb-ed06-4c65-b87d-74f8f825457d
Apr 14 04:30:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-15c2efab-5cec-4e77-99ec-7c35a91c11f8 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "a0a20aeb-ed06-4c65-b87d-74f8f825457d" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 5.451s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:24 user nova-compute[71317]: INFO nova.compute.manager [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Terminating instance
Apr 14 04:30:24 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG nova.compute.manager [req-0d146742-488c-4e26-acc1-9fd58cefe2f3 req-55777f89-1ef1-446a-96b9-249c91d06a4c service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Received event network-vif-unplugged-dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0d146742-488c-4e26-acc1-9fd58cefe2f3 req-55777f89-1ef1-446a-96b9-249c91d06a4c service nova] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0d146742-488c-4e26-acc1-9fd58cefe2f3 req-55777f89-1ef1-446a-96b9-249c91d06a4c service nova] Lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0d146742-488c-4e26-acc1-9fd58cefe2f3 req-55777f89-1ef1-446a-96b9-249c91d06a4c service nova] Lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG nova.compute.manager [req-0d146742-488c-4e26-acc1-9fd58cefe2f3 req-55777f89-1ef1-446a-96b9-249c91d06a4c service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] No waiting events found dispatching network-vif-unplugged-dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:30:24 user nova-compute[71317]: DEBUG nova.compute.manager [req-0d146742-488c-4e26-acc1-9fd58cefe2f3 req-55777f89-1ef1-446a-96b9-249c91d06a4c service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Received event network-vif-unplugged-dd12687e-ca27-4fba-9653-4fa1af3608a8 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:30:25 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Instance destroyed successfully.
Apr 14 04:30:25 user nova-compute[71317]: DEBUG nova.objects.instance [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lazy-loading 'resources' on Instance uuid f593ceb0-de44-490b-b365-3a05a3acd066 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:30:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:23:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-141729612',display_name='tempest-SnapshotDataIntegrityTests-server-141729612',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-141729612',id=22,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOjIW2Ycb+mq2tNzF5oSfsdR8libq7W1H6v9ImKoXEDVO168nnN9ZHy7JgCycEV+p/J22r0BDP5EimQKsNhepd1EFrAa2ljn1iDdZ9Zn90UYL0Ok/xoK1pzqla/0GV9JdQ==',key_name='tempest-SnapshotDataIntegrityTests-881844326',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:23:51Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='39731dbb51484eff890a74b07241ba75',ramdisk_id='',reservation_id='r-kqgpo0fv',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-SnapshotDataIntegrityTests-502963555',owner_user_name='tempest-SnapshotDataIntegrityTests-502963555-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:23:52Z,user_data=None,user_id='0d3c1b814ec240279646878fd090f5f2',uuid=f593ceb0-de44-490b-b365-3a05a3acd066,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "address": "fa:16:3e:17:59:61", "network": {"id": "694b7c0a-cb94-4b83-99aa-03c94c049086", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-383837474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "39731dbb51484eff890a74b07241ba75", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd12687e-ca", "ovs_interfaceid": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:30:25 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Converting VIF {"id": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "address": "fa:16:3e:17:59:61", "network": {"id": "694b7c0a-cb94-4b83-99aa-03c94c049086", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-383837474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "39731dbb51484eff890a74b07241ba75", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd12687e-ca", "ovs_interfaceid": "dd12687e-ca27-4fba-9653-4fa1af3608a8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:30:25 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:17:59:61,bridge_name='br-int',has_traffic_filtering=True,id=dd12687e-ca27-4fba-9653-4fa1af3608a8,network=Network(694b7c0a-cb94-4b83-99aa-03c94c049086),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd12687e-ca') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:30:25 user nova-compute[71317]: DEBUG os_vif [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:17:59:61,bridge_name='br-int',has_traffic_filtering=True,id=dd12687e-ca27-4fba-9653-4fa1af3608a8,network=Network(694b7c0a-cb94-4b83-99aa-03c94c049086),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd12687e-ca') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:30:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapdd12687e-ca, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:30:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:30:25 user nova-compute[71317]: INFO os_vif [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:17:59:61,bridge_name='br-int',has_traffic_filtering=True,id=dd12687e-ca27-4fba-9653-4fa1af3608a8,network=Network(694b7c0a-cb94-4b83-99aa-03c94c049086),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd12687e-ca')
Apr 14 04:30:25 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Deleting instance files /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066_del
Apr 14 04:30:25 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Deletion of /opt/stack/data/nova/instances/f593ceb0-de44-490b-b365-3a05a3acd066_del complete
Apr 14 04:30:25 user nova-compute[71317]: INFO nova.compute.manager [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Took 0.97 seconds to destroy the instance on the hypervisor.
Apr 14 04:30:25 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:30:25 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:30:25 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:30:26 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:30:26 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Took 0.89 seconds to deallocate network for instance.
Apr 14 04:30:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-38032d3e-dad7-4155-ab22-32b57f7e7339 req-f56e164b-ce20-426a-b76b-1636022f5328 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Received event network-vif-deleted-dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:26 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:30:26 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:30:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.367s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:26 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Deleted allocations for instance f593ceb0-de44-490b-b365-3a05a3acd066
Apr 14 04:30:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bf7c465d-12d1-4a96-8f27-013e390c772c tempest-SnapshotDataIntegrityTests-502963555 tempest-SnapshotDataIntegrityTests-502963555-project-member] Lock "f593ceb0-de44-490b-b365-3a05a3acd066" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.492s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-19b7fdfc-080c-46bd-bdcc-cd7ebdd39e74 req-87d301d8-ab7f-460e-8165-45d98d623769 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Received event network-vif-plugged-dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-19b7fdfc-080c-46bd-bdcc-cd7ebdd39e74 req-87d301d8-ab7f-460e-8165-45d98d623769 service nova] Acquiring lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-19b7fdfc-080c-46bd-bdcc-cd7ebdd39e74 req-87d301d8-ab7f-460e-8165-45d98d623769 service nova] Lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-19b7fdfc-080c-46bd-bdcc-cd7ebdd39e74 req-87d301d8-ab7f-460e-8165-45d98d623769 service nova] Lock "f593ceb0-de44-490b-b365-3a05a3acd066-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-19b7fdfc-080c-46bd-bdcc-cd7ebdd39e74 req-87d301d8-ab7f-460e-8165-45d98d623769 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] No waiting events found dispatching network-vif-plugged-dd12687e-ca27-4fba-9653-4fa1af3608a8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:30:26 user nova-compute[71317]: WARNING nova.compute.manager [req-19b7fdfc-080c-46bd-bdcc-cd7ebdd39e74 req-87d301d8-ab7f-460e-8165-45d98d623769 service nova] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Received unexpected event network-vif-plugged-dd12687e-ca27-4fba-9653-4fa1af3608a8 for instance with vm_state deleted and task_state None.
Apr 14 04:30:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:34 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446619.216571, a0a20aeb-ed06-4c65-b87d-74f8f825457d => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:30:34 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] VM Stopped (Lifecycle Event)
Apr 14 04:30:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e6038d5c-cc7a-43f5-82f8-65ce8aa413fe None None] [instance: a0a20aeb-ed06-4c65-b87d-74f8f825457d] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:30:34 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:30:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:30:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:30:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:30:34 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:30:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:35 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updating instance_info_cache with network_info: [{"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:30:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:30:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:30:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:30:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:30:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:30:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:30:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:30:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:30:36 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:30:37 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:30:38 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:30:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:30:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446625.2065003, f593ceb0-de44-490b-b365-3a05a3acd066 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:30:40 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] VM Stopped (Lifecycle Event)
Apr 14 04:30:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG nova.compute.manager [None req-83ba6560-0c82-47d0-94a8-6b2d955246d1 None None] [instance: f593ceb0-de44-490b-b365-3a05a3acd066] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdg (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:30:41 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:30:41 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7647MB free_disk=26.31741714477539GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance af7befec-3f77-4eb7-be53-d1b1e791713e actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 19a166de-f0c9-43f2-8c88-585e0d5d22cc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 390ca845-5660-49d0-bd8a-fcfc39176ca5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:30:41 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:30:42 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:30:42 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:30:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:30:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.476s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "ca085872-69ff-41e4-be13-ffaa9a7cca82" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:30:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:50 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:30:50 user nova-compute[71317]: INFO nova.compute.claims [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Claim successful on node user
Apr 14 04:30:50 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:30:50 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:30:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.643s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:30:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:30:51 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:30:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:30:51 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:30:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:30:51 user nova-compute[71317]: INFO nova.virt.block_device [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Booting with volume snapshot f5cc0b68-6502-4a55-96a0-69a8f27c433c at /dev/vda
Apr 14 04:30:51 user nova-compute[71317]: DEBUG nova.policy [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bcf79de7da27495dbfd14c556b5b54c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '693d95858c624feaa60114aa89082186', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:30:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Successfully created port: f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:30:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Successfully updated port: f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:30:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "refresh_cache-ca085872-69ff-41e4-be13-ffaa9a7cca82" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:30:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquired lock "refresh_cache-ca085872-69ff-41e4-be13-ffaa9a7cca82" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:30:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:30:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-c2e63357-6e4e-4076-a724-e694c9a40e39 req-1570cef3-0fcc-40d1-b614-fa6b38e88aee service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Received event network-changed-f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-c2e63357-6e4e-4076-a724-e694c9a40e39 req-1570cef3-0fcc-40d1-b614-fa6b38e88aee service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Refreshing instance network info cache due to event network-changed-f010668b-3ce4-4f88-a116-dd9a67663985. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:30:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c2e63357-6e4e-4076-a724-e694c9a40e39 req-1570cef3-0fcc-40d1-b614-fa6b38e88aee service nova] Acquiring lock "refresh_cache-ca085872-69ff-41e4-be13-ffaa9a7cca82" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:30:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:30:53 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Updating instance_info_cache with network_info: [{"id": "f010668b-3ce4-4f88-a116-dd9a67663985", "address": "fa:16:3e:df:f8:42", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf010668b-3c", "ovs_interfaceid": "f010668b-3ce4-4f88-a116-dd9a67663985", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:30:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Releasing lock "refresh_cache-ca085872-69ff-41e4-be13-ffaa9a7cca82" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:30:53 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Instance network_info: |[{"id": "f010668b-3ce4-4f88-a116-dd9a67663985", "address": "fa:16:3e:df:f8:42", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf010668b-3c", "ovs_interfaceid": "f010668b-3ce4-4f88-a116-dd9a67663985", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:30:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c2e63357-6e4e-4076-a724-e694c9a40e39 req-1570cef3-0fcc-40d1-b614-fa6b38e88aee service nova] Acquired lock "refresh_cache-ca085872-69ff-41e4-be13-ffaa9a7cca82" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:30:53 user nova-compute[71317]: DEBUG nova.network.neutron [req-c2e63357-6e4e-4076-a724-e694c9a40e39 req-1570cef3-0fcc-40d1-b614-fa6b38e88aee service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Refreshing network info cache for port f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:30:53 user nova-compute[71317]: DEBUG nova.network.neutron [req-c2e63357-6e4e-4076-a724-e694c9a40e39 req-1570cef3-0fcc-40d1-b614-fa6b38e88aee service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Updated VIF entry in instance network info cache for port f010668b-3ce4-4f88-a116-dd9a67663985. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:30:53 user nova-compute[71317]: DEBUG nova.network.neutron [req-c2e63357-6e4e-4076-a724-e694c9a40e39 req-1570cef3-0fcc-40d1-b614-fa6b38e88aee service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Updating instance_info_cache with network_info: [{"id": "f010668b-3ce4-4f88-a116-dd9a67663985", "address": "fa:16:3e:df:f8:42", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf010668b-3c", "ovs_interfaceid": "f010668b-3ce4-4f88-a116-dd9a67663985", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:30:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c2e63357-6e4e-4076-a724-e694c9a40e39 req-1570cef3-0fcc-40d1-b614-fa6b38e88aee service nova] Releasing lock "refresh_cache-ca085872-69ff-41e4-be13-ffaa9a7cca82" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:30:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:30:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG os_brick.utils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f03917ab-b9f0-46e0-a82b-b83e5e1d73dd]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ae7f3bd-7dd9-4ce1-9213-04794d5df45d]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb2ab3c8-b60a-48fb-bf77-64d31a9e377e]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[85fb649d-8210-4488-aa14-f572bbe6f0a9]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:30:57 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:30:57 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[92afcc04-eae2-4448-924f-3d6849f5b42b]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG os_brick.utils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== get_connector_properties: return (124ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:30:57 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Updating existing volume attachment record: 0f9216d9-463d-4546-b8b1-f30bf5bac3d6 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:30:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-43ea00a8-a8c5-4ea9-a2a3-68e3458162f4 req-e9701895-09ed-4669-af84-d0a85ee7115a service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-changed-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:30:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-43ea00a8-a8c5-4ea9-a2a3-68e3458162f4 req-e9701895-09ed-4669-af84-d0a85ee7115a service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Refreshing instance network info cache due to event network-changed-e3868733-d538-4469-9a3e-c02014ec9066. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:30:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-43ea00a8-a8c5-4ea9-a2a3-68e3458162f4 req-e9701895-09ed-4669-af84-d0a85ee7115a service nova] Acquiring lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:30:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-43ea00a8-a8c5-4ea9-a2a3-68e3458162f4 req-e9701895-09ed-4669-af84-d0a85ee7115a service nova] Acquired lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:30:58 user nova-compute[71317]: DEBUG nova.network.neutron [req-43ea00a8-a8c5-4ea9-a2a3-68e3458162f4 req-e9701895-09ed-4669-af84-d0a85ee7115a service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Refreshing network info cache for port e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:30:59 user nova-compute[71317]: INFO nova.compute.manager [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Detaching volume 2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:30:59 user nova-compute[71317]: INFO nova.virt.block_device [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Attempting to driver detach volume 2ea04249-09da-471f-a8b5-89a901381329 from mountpoint /dev/vdb
Apr 14 04:30:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Attempting to detach device vdb from instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:30:59 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:30:59 user nova-compute[71317]:   <source dev="/dev/sdg"/>
Apr 14 04:30:59 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:30:59 user nova-compute[71317]:   <serial>2ea04249-09da-471f-a8b5-89a901381329</serial>
Apr 14 04:30:59 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:30:59 user nova-compute[71317]: </disk>
Apr 14 04:30:59 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:30:59 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Successfully detached device vdb from instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c from the persistent domain config.
Apr 14 04:30:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:30:59 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:30:59 user nova-compute[71317]:   <source dev="/dev/sdg"/>
Apr 14 04:30:59 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:30:59 user nova-compute[71317]:   <serial>2ea04249-09da-471f-a8b5-89a901381329</serial>
Apr 14 04:30:59 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:30:59 user nova-compute[71317]: </disk>
Apr 14 04:30:59 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-43ea00a8-a8c5-4ea9-a2a3-68e3458162f4 req-e9701895-09ed-4669-af84-d0a85ee7115a service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updated VIF entry in instance network info cache for port e3868733-d538-4469-9a3e-c02014ec9066. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-43ea00a8-a8c5-4ea9-a2a3-68e3458162f4 req-e9701895-09ed-4669-af84-d0a85ee7115a service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updating instance_info_cache with network_info: [{"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-43ea00a8-a8c5-4ea9-a2a3-68e3458162f4 req-e9701895-09ed-4669-af84-d0a85ee7115a service nova] Releasing lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446659.8788242, cab84e5c-99ba-4e61-bf68-0a02b31ded5c => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:30:59 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Successfully detached device vdb from instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c from the live domain config.
Apr 14 04:30:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dWfGZPuJ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:30:59 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:30:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[03d20995-89ea-420e-8d87-86de2688db5b]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:30:59 user nova-compute[71317]: No targets found.
Apr 14 04:30:59 user nova-compute[71317]: iSNS:
Apr 14 04:30:59 user nova-compute[71317]: No targets found.
Apr 14 04:30:59 user nova-compute[71317]: STATIC:
Apr 14 04:30:59 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329
Apr 14 04:30:59 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:30:59 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:30:59 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895
Apr 14 04:30:59 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:30:59 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:30:59 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:30:59 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:30:59 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:30:59 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4
Apr 14 04:30:59 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:30:59 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:30:59 user nova-compute[71317]: FIRMWARE:
Apr 14 04:30:59 user nova-compute[71317]: No targets found.
Apr 14 04:30:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:30:59 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:30:59 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c35784e9-76fe-4bd0-b22b-886dfad25d6d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c8808d9-9ea3-4087-a54f-58108fe8cb6a]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:30:59 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:30:59 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:30:59 user nova-compute[71317]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:30:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:30:59 user nova-compute[71317]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 (non-flash)
Apr 14 04:30:59 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:30:59 user nova-compute[71317]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:30:59 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d19745e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329'): ({'sdg'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Removing single pathed devices sdg {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:30:59 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe1239f8-ac52-45af-b4dd-c60e66dba717]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e99b6b7-ce27-4c3d-8ba1-38823e8a5c53]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Flushing IO for device /dev/sdg {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ad2e2c6-cff3-4f5a-a908-7aaf4587d2c0]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.043s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[85101bed-887f-496f-8b1f-804e24ea0c86]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] SCSI volumes sdg have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[01406865-95ec-43bf-a644-f2fa9f4fe3a1]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --logout" returned: 0 in 0.070s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea3194ad-01f8-4930-be30-df71c2b0c919]: (4, ('Logging out of session [sid: 33, target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329, portal: 172.16.0.220,3260]\nLogout of [sid: 33, target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 33, target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329, portal: 172.16.0.220,3260]
Apr 14 04:31:00 user nova-compute[71317]: Logout of [sid: 33, target: iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329, portal: 172.16.0.220,3260] successful.
Apr 14 04:31:00 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2ea04249-09da-471f-a8b5-89a901381329 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.033s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c186f05a-ead2-47ca-9987-409b5b3760ab]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.343s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] <== disconnect_volume: return (345ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-748fac49-d90f-4cfc-bb95-30c26aad413d req-61e74250-fe57-433b-bc21-a5ac64bc493d service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-changed-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-748fac49-d90f-4cfc-bb95-30c26aad413d req-61e74250-fe57-433b-bc21-a5ac64bc493d service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Refreshing instance network info cache due to event network-changed-e3868733-d538-4469-9a3e-c02014ec9066. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-748fac49-d90f-4cfc-bb95-30c26aad413d req-61e74250-fe57-433b-bc21-a5ac64bc493d service nova] Acquiring lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-748fac49-d90f-4cfc-bb95-30c26aad413d req-61e74250-fe57-433b-bc21-a5ac64bc493d service nova] Acquired lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:31:00 user nova-compute[71317]: DEBUG nova.network.neutron [req-748fac49-d90f-4cfc-bb95-30c26aad413d req-61e74250-fe57-433b-bc21-a5ac64bc493d service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Refreshing network info cache for port e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:31:01 user nova-compute[71317]: DEBUG nova.objects.instance [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'flavor' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:31:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-71f502c4-224b-46fa-a25f-17ec0650a7df tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.787s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:01 user nova-compute[71317]: DEBUG nova.network.neutron [req-748fac49-d90f-4cfc-bb95-30c26aad413d req-61e74250-fe57-433b-bc21-a5ac64bc493d service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updated VIF entry in instance network info cache for port e3868733-d538-4469-9a3e-c02014ec9066. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:31:01 user nova-compute[71317]: DEBUG nova.network.neutron [req-748fac49-d90f-4cfc-bb95-30c26aad413d req-61e74250-fe57-433b-bc21-a5ac64bc493d service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updating instance_info_cache with network_info: [{"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:31:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-748fac49-d90f-4cfc-bb95-30c26aad413d req-61e74250-fe57-433b-bc21-a5ac64bc493d service nova] Releasing lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:31:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:31:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:31:01 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Creating image(s)
Apr 14 04:31:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:31:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Ensure instance console log exists: /opt/stack/data/nova/instances/ca085872-69ff-41e4-be13-ffaa9a7cca82/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:31:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Start _get_guest_xml network_info=[{"id": "f010668b-3ce4-4f88-a116-dd9a67663985", "address": "fa:16:3e:df:f8:42", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf010668b-3c", "ovs_interfaceid": "f010668b-3ce4-4f88-a116-dd9a67663985", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='d41d8cd98f00b204e9800998ecf8427e',container_format='bare',created_at=2023-04-14T04:30:47Z,direct_url=<?>,disk_format='qcow2',id=37c9a7e6-d203-4290-ba7e-602fdff12012,min_disk=1,min_ram=0,name='tempest-TestVolumeBootPatternsnapshot-706995129',owner='693d95858c624feaa60114aa89082186',properties=ImageMetaProps,protected=<?>,size=0,status='active',tags=<?>,updated_at=2023-04-14T04:30:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': True, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qBXmfngt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'ca085872-69ff-41e4-be13-ffaa9a7cca82', 'attached_at': '', 'detached_at': '', 'volume_id': 'e9e63107-a32f-4221-964d-9ee193d260fb', 'serial': 'e9e63107-a32f-4221-964d-9ee193d260fb'}, 'attachment_id': '0f9216d9-463d-4546-b8b1-f30bf5bac3d6', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:31:01 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:31:01 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='d41d8cd98f00b204e9800998ecf8427e',container_format='bare',created_at=2023-04-14T04:30:47Z,direct_url=<?>,disk_format='qcow2',id=37c9a7e6-d203-4290-ba7e-602fdff12012,min_disk=1,min_ram=0,name='tempest-TestVolumeBootPatternsnapshot-706995129',owner='693d95858c624feaa60114aa89082186',properties=ImageMetaProps,protected=<?>,size=0,status='active',tags=<?>,updated_at=2023-04-14T04:30:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qBXmfngt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:31:02 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:31:02 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:31:02 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260" returned: 21 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9baa517-c33d-44a9-8f74-5e254485bddc]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:31:02 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2c357e3-2d65-465a-a15a-46fe93cd3bbf]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb] added
Apr 14 04:31:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[288eeee4-e5de-4267-9492-860beec62175]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2eceb2e8-d29c-455a-a23d-fc5cff5271db]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qBXmfngt {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qBXmfngt" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[de82fadd-76cd-4bb8-b319-d0a8df549a39]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'qBXmfngt'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ff839396-58d0-4c16-ab52-364414fb4221]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[348878de-4aa1-46a0-9204-ddab986656d0]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:31:02 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:31:02 user nova-compute[71317]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:31:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:31:02 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:31:02 user nova-compute[71317]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:31:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --login" returned: 0 in 0.063s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ee644e3-d8a6-4bbd-8508-db1a1af4d8d0]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb, portal: 172.16.0.220,3260]
Apr 14 04:31:02 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb, portal: 172.16.0.220,3260] successful.
Apr 14 04:31:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[434109de-41f5-4428-90d0-9e08c56e96bd]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b0a7534-a53e-4ce3-a452-b3cf9e1d23da]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:31:02 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:31:02 user nova-compute[71317]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:31:02 user nova-compute[71317]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)
Apr 14 04:31:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:31:02 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:31:02 user nova-compute[71317]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:31:02 user nova-compute[71317]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)
Apr 14 04:31:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.288s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] HCTL ('35', '-', '-', 0) found on session 42 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c434c664-c739-484e-bc98-f5cd016298c8]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Searching for a device in session 42 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:31:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qBXmfngt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.321s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== connect_volume: return (1322ms) {'type': 'block', 'scsi_wwn': '23831356138633730', 'path': '/dev/sdd'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23831356138633730', 'path': '/dev/sdd'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:30:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-453223105',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-453223105',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-453223105',id=33,image_ref='37c9a7e6-d203-4290-ba7e-602fdff12012',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-7vc9kziz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='member,reader',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-1780160917',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member',image_root_device_name='/dev/vda',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:30:51Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=ca085872-69ff-41e4-be13-ffaa9a7cca82,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f010668b-3ce4-4f88-a116-dd9a67663985", "address": "fa:16:3e:df:f8:42", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf010668b-3c", "ovs_interfaceid": "f010668b-3ce4-4f88-a116-dd9a67663985", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "f010668b-3ce4-4f88-a116-dd9a67663985", "address": "fa:16:3e:df:f8:42", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf010668b-3c", "ovs_interfaceid": "f010668b-3ce4-4f88-a116-dd9a67663985", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:df:f8:42,bridge_name='br-int',has_traffic_filtering=True,id=f010668b-3ce4-4f88-a116-dd9a67663985,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf010668b-3c') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lazy-loading 'pci_devices' on Instance uuid ca085872-69ff-41e4-be13-ffaa9a7cca82 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:31:03 user nova-compute[71317]:   <uuid>ca085872-69ff-41e4-be13-ffaa9a7cca82</uuid>
Apr 14 04:31:03 user nova-compute[71317]:   <name>instance-00000021</name>
Apr 14 04:31:03 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:31:03 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:31:03 user nova-compute[71317]:   <metadata>
Apr 14 04:31:03 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:31:03 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:31:03 user nova-compute[71317]:       <nova:name>tempest-TestVolumeBootPattern-image-snapshot-server-453223105</nova:name>
Apr 14 04:31:03 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:31:01</nova:creationTime>
Apr 14 04:31:03 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:31:03 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:31:03 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:31:03 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:31:03 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:31:03 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:31:03 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:31:03 user nova-compute[71317]:       <nova:owner>
Apr 14 04:31:03 user nova-compute[71317]:         <nova:user uuid="bcf79de7da27495dbfd14c556b5b54c7">tempest-TestVolumeBootPattern-1780160917-project-member</nova:user>
Apr 14 04:31:03 user nova-compute[71317]:         <nova:project uuid="693d95858c624feaa60114aa89082186">tempest-TestVolumeBootPattern-1780160917</nova:project>
Apr 14 04:31:03 user nova-compute[71317]:       </nova:owner>
Apr 14 04:31:03 user nova-compute[71317]:       <nova:root type="image" uuid="37c9a7e6-d203-4290-ba7e-602fdff12012"/>
Apr 14 04:31:03 user nova-compute[71317]:       <nova:ports>
Apr 14 04:31:03 user nova-compute[71317]:         <nova:port uuid="f010668b-3ce4-4f88-a116-dd9a67663985">
Apr 14 04:31:03 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 04:31:03 user nova-compute[71317]:         </nova:port>
Apr 14 04:31:03 user nova-compute[71317]:       </nova:ports>
Apr 14 04:31:03 user nova-compute[71317]:     </nova:instance>
Apr 14 04:31:03 user nova-compute[71317]:   </metadata>
Apr 14 04:31:03 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:31:03 user nova-compute[71317]:     <system>
Apr 14 04:31:03 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:31:03 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:31:03 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:31:03 user nova-compute[71317]:       <entry name="serial">ca085872-69ff-41e4-be13-ffaa9a7cca82</entry>
Apr 14 04:31:03 user nova-compute[71317]:       <entry name="uuid">ca085872-69ff-41e4-be13-ffaa9a7cca82</entry>
Apr 14 04:31:03 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:31:03 user nova-compute[71317]:     </system>
Apr 14 04:31:03 user nova-compute[71317]:   </sysinfo>
Apr 14 04:31:03 user nova-compute[71317]:   <os>
Apr 14 04:31:03 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:31:03 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:31:03 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:31:03 user nova-compute[71317]:   </os>
Apr 14 04:31:03 user nova-compute[71317]:   <features>
Apr 14 04:31:03 user nova-compute[71317]:     <acpi/>
Apr 14 04:31:03 user nova-compute[71317]:     <apic/>
Apr 14 04:31:03 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:31:03 user nova-compute[71317]:   </features>
Apr 14 04:31:03 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:31:03 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:31:03 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:31:03 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:31:03 user nova-compute[71317]:   </clock>
Apr 14 04:31:03 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:31:03 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:31:03 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:31:03 user nova-compute[71317]:   </cpu>
Apr 14 04:31:03 user nova-compute[71317]:   <devices>
Apr 14 04:31:03 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:31:03 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:31:03 user nova-compute[71317]:       <source dev="/dev/sdd"/>
Apr 14 04:31:03 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:31:03 user nova-compute[71317]:       <serial>e9e63107-a32f-4221-964d-9ee193d260fb</serial>
Apr 14 04:31:03 user nova-compute[71317]:     </disk>
Apr 14 04:31:03 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:31:03 user nova-compute[71317]:       <mac address="fa:16:3e:df:f8:42"/>
Apr 14 04:31:03 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:31:03 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:31:03 user nova-compute[71317]:       <target dev="tapf010668b-3c"/>
Apr 14 04:31:03 user nova-compute[71317]:     </interface>
Apr 14 04:31:03 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:31:03 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/ca085872-69ff-41e4-be13-ffaa9a7cca82/console.log" append="off"/>
Apr 14 04:31:03 user nova-compute[71317]:     </serial>
Apr 14 04:31:03 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:31:03 user nova-compute[71317]:     <video>
Apr 14 04:31:03 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:31:03 user nova-compute[71317]:     </video>
Apr 14 04:31:03 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:31:03 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:31:03 user nova-compute[71317]:     </rng>
Apr 14 04:31:03 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:31:03 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:31:03 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:31:03 user nova-compute[71317]:     </memballoon>
Apr 14 04:31:03 user nova-compute[71317]:   </devices>
Apr 14 04:31:03 user nova-compute[71317]: </domain>
Apr 14 04:31:03 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:30:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-453223105',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-453223105',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-453223105',id=33,image_ref='37c9a7e6-d203-4290-ba7e-602fdff12012',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-7vc9kziz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='member,reader',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-1780160917',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member',image_root_device_name='/dev/vda',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:30:51Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=ca085872-69ff-41e4-be13-ffaa9a7cca82,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f010668b-3ce4-4f88-a116-dd9a67663985", "address": "fa:16:3e:df:f8:42", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf010668b-3c", "ovs_interfaceid": "f010668b-3ce4-4f88-a116-dd9a67663985", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "f010668b-3ce4-4f88-a116-dd9a67663985", "address": "fa:16:3e:df:f8:42", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf010668b-3c", "ovs_interfaceid": "f010668b-3ce4-4f88-a116-dd9a67663985", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:df:f8:42,bridge_name='br-int',has_traffic_filtering=True,id=f010668b-3ce4-4f88-a116-dd9a67663985,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf010668b-3c') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG os_vif [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:df:f8:42,bridge_name='br-int',has_traffic_filtering=True,id=f010668b-3ce4-4f88-a116-dd9a67663985,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf010668b-3c') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapf010668b-3c, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapf010668b-3c, col_values=(('external_ids', {'iface-id': 'f010668b-3ce4-4f88-a116-dd9a67663985', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:df:f8:42', 'vm-uuid': 'ca085872-69ff-41e4-be13-ffaa9a7cca82'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:03 user nova-compute[71317]: INFO os_vif [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:df:f8:42,bridge_name='br-int',has_traffic_filtering=True,id=f010668b-3ce4-4f88-a116-dd9a67663985,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf010668b-3c')
Apr 14 04:31:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:31:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] No VIF found with MAC fa:16:3e:df:f8:42, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:31:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-6b938bf4-d7b6-4280-b0ed-924c7ec901d8 req-df3f7498-8358-46bc-958d-68f87925f400 service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Received event network-vif-plugged-f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6b938bf4-d7b6-4280-b0ed-924c7ec901d8 req-df3f7498-8358-46bc-958d-68f87925f400 service nova] Acquiring lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6b938bf4-d7b6-4280-b0ed-924c7ec901d8 req-df3f7498-8358-46bc-958d-68f87925f400 service nova] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6b938bf4-d7b6-4280-b0ed-924c7ec901d8 req-df3f7498-8358-46bc-958d-68f87925f400 service nova] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-6b938bf4-d7b6-4280-b0ed-924c7ec901d8 req-df3f7498-8358-46bc-958d-68f87925f400 service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] No waiting events found dispatching network-vif-plugged-f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:31:04 user nova-compute[71317]: WARNING nova.compute.manager [req-6b938bf4-d7b6-4280-b0ed-924c7ec901d8 req-df3f7498-8358-46bc-958d-68f87925f400 service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Received unexpected event network-vif-plugged-f010668b-3ce4-4f88-a116-dd9a67663985 for instance with vm_state building and task_state spawning.
Apr 14 04:31:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:06 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446666.9462392, ca085872-69ff-41e4-be13-ffaa9a7cca82 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:31:06 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] VM Resumed (Lifecycle Event)
Apr 14 04:31:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:31:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:31:06 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Instance spawned successfully.
Apr 14 04:31:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:31:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:31:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:31:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:31:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:31:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-e029e86f-5df3-4150-a861-0cbcc8d921d7 req-e75457a2-fdbc-45d4-b07d-20bf6bd0e81e service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Received event network-vif-plugged-f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e029e86f-5df3-4150-a861-0cbcc8d921d7 req-e75457a2-fdbc-45d4-b07d-20bf6bd0e81e service nova] Acquiring lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e029e86f-5df3-4150-a861-0cbcc8d921d7 req-e75457a2-fdbc-45d4-b07d-20bf6bd0e81e service nova] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e029e86f-5df3-4150-a861-0cbcc8d921d7 req-e75457a2-fdbc-45d4-b07d-20bf6bd0e81e service nova] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-e029e86f-5df3-4150-a861-0cbcc8d921d7 req-e75457a2-fdbc-45d4-b07d-20bf6bd0e81e service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] No waiting events found dispatching network-vif-plugged-f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:31:06 user nova-compute[71317]: WARNING nova.compute.manager [req-e029e86f-5df3-4150-a861-0cbcc8d921d7 req-e75457a2-fdbc-45d4-b07d-20bf6bd0e81e service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Received unexpected event network-vif-plugged-f010668b-3ce4-4f88-a116-dd9a67663985 for instance with vm_state building and task_state spawning.
Apr 14 04:31:07 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:31:07 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446666.9463594, ca085872-69ff-41e4-be13-ffaa9a7cca82 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:31:07 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] VM Started (Lifecycle Event)
Apr 14 04:31:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:31:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:31:07 user nova-compute[71317]: INFO nova.compute.manager [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Took 5.06 seconds to spawn the instance on the hypervisor.
Apr 14 04:31:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:31:07 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:31:07 user nova-compute[71317]: INFO nova.compute.manager [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Took 16.74 seconds to build instance.
Apr 14 04:31:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c1c3b813-732c-43ca-93eb-102f507975fb tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.834s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:09 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:31:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:09 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:31:09 user nova-compute[71317]: INFO nova.compute.claims [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Claim successful on node user
Apr 14 04:31:10 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.503s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:31:10 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:31:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG nova.policy [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e35b3ae371cf4dd4b21e2999e58cd2f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c14c332dfcc44b49ddf3cb482dd0d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:31:10 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Creating image(s)
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "/opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "/opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "/opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.010s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.181s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk 1073741824" returned: 0 in 0.054s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.242s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.160s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Checking if we can resize image /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:31:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:11 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Cannot resize image /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:31:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'migration_context' on Instance uuid 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:31:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:31:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Ensure instance console log exists: /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:31:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:11 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Successfully created port: bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Successfully updated port: bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquired lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-25761e43-cb69-4c40-bc44-d67644af0713 req-7fd7bb17-8cbb-453c-b460-0e2d91340c7e service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Received event network-changed-bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-25761e43-cb69-4c40-bc44-d67644af0713 req-7fd7bb17-8cbb-453c-b460-0e2d91340c7e service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Refreshing instance network info cache due to event network-changed-bc83f476-5176-4301-8272-80034f4a0daf. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-25761e43-cb69-4c40-bc44-d67644af0713 req-7fd7bb17-8cbb-453c-b460-0e2d91340c7e service nova] Acquiring lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.network.neutron [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Updating instance_info_cache with network_info: [{"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Releasing lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Instance network_info: |[{"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-25761e43-cb69-4c40-bc44-d67644af0713 req-7fd7bb17-8cbb-453c-b460-0e2d91340c7e service nova] Acquired lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.network.neutron [req-25761e43-cb69-4c40-bc44-d67644af0713 req-7fd7bb17-8cbb-453c-b460-0e2d91340c7e service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Refreshing network info cache for port bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Start _get_guest_xml network_info=[{"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:31:12 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:31:12 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:31:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-496557296',display_name='tempest-ServerActionsTestJSON-server-496557296',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-496557296',id=34,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-olssq35z',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:31:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:69:b3:83,bridge_name='br-int',has_traffic_filtering=True,id=bc83f476-5176-4301-8272-80034f4a0daf,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbc83f476-51') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.objects.instance [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'pci_devices' on Instance uuid 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:31:12 user nova-compute[71317]:   <uuid>123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c</uuid>
Apr 14 04:31:12 user nova-compute[71317]:   <name>instance-00000022</name>
Apr 14 04:31:12 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:31:12 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:31:12 user nova-compute[71317]:   <metadata>
Apr 14 04:31:12 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:31:12 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:31:12 user nova-compute[71317]:       <nova:name>tempest-ServerActionsTestJSON-server-496557296</nova:name>
Apr 14 04:31:12 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:31:12</nova:creationTime>
Apr 14 04:31:12 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:31:12 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:31:12 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:31:12 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:31:12 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:31:12 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:31:12 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:31:12 user nova-compute[71317]:       <nova:owner>
Apr 14 04:31:12 user nova-compute[71317]:         <nova:user uuid="e35b3ae371cf4dd4b21e2999e58cd2f4">tempest-ServerActionsTestJSON-1048786641-project-member</nova:user>
Apr 14 04:31:12 user nova-compute[71317]:         <nova:project uuid="8c14c332dfcc44b49ddf3cb482dd0d4c">tempest-ServerActionsTestJSON-1048786641</nova:project>
Apr 14 04:31:12 user nova-compute[71317]:       </nova:owner>
Apr 14 04:31:12 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:31:12 user nova-compute[71317]:       <nova:ports>
Apr 14 04:31:12 user nova-compute[71317]:         <nova:port uuid="bc83f476-5176-4301-8272-80034f4a0daf">
Apr 14 04:31:12 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Apr 14 04:31:12 user nova-compute[71317]:         </nova:port>
Apr 14 04:31:12 user nova-compute[71317]:       </nova:ports>
Apr 14 04:31:12 user nova-compute[71317]:     </nova:instance>
Apr 14 04:31:12 user nova-compute[71317]:   </metadata>
Apr 14 04:31:12 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:31:12 user nova-compute[71317]:     <system>
Apr 14 04:31:12 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:31:12 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:31:12 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:31:12 user nova-compute[71317]:       <entry name="serial">123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c</entry>
Apr 14 04:31:12 user nova-compute[71317]:       <entry name="uuid">123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c</entry>
Apr 14 04:31:12 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:31:12 user nova-compute[71317]:     </system>
Apr 14 04:31:12 user nova-compute[71317]:   </sysinfo>
Apr 14 04:31:12 user nova-compute[71317]:   <os>
Apr 14 04:31:12 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:31:12 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:31:12 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:31:12 user nova-compute[71317]:   </os>
Apr 14 04:31:12 user nova-compute[71317]:   <features>
Apr 14 04:31:12 user nova-compute[71317]:     <acpi/>
Apr 14 04:31:12 user nova-compute[71317]:     <apic/>
Apr 14 04:31:12 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:31:12 user nova-compute[71317]:   </features>
Apr 14 04:31:12 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:31:12 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:31:12 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:31:12 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:31:12 user nova-compute[71317]:   </clock>
Apr 14 04:31:12 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:31:12 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:31:12 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:31:12 user nova-compute[71317]:   </cpu>
Apr 14 04:31:12 user nova-compute[71317]:   <devices>
Apr 14 04:31:12 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:31:12 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:31:12 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk"/>
Apr 14 04:31:12 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:31:12 user nova-compute[71317]:     </disk>
Apr 14 04:31:12 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:31:12 user nova-compute[71317]:       <mac address="fa:16:3e:69:b3:83"/>
Apr 14 04:31:12 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:31:12 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:31:12 user nova-compute[71317]:       <target dev="tapbc83f476-51"/>
Apr 14 04:31:12 user nova-compute[71317]:     </interface>
Apr 14 04:31:12 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:31:12 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/console.log" append="off"/>
Apr 14 04:31:12 user nova-compute[71317]:     </serial>
Apr 14 04:31:12 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:31:12 user nova-compute[71317]:     <video>
Apr 14 04:31:12 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:31:12 user nova-compute[71317]:     </video>
Apr 14 04:31:12 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:31:12 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:31:12 user nova-compute[71317]:     </rng>
Apr 14 04:31:12 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:31:12 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:31:12 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:31:12 user nova-compute[71317]:     </memballoon>
Apr 14 04:31:12 user nova-compute[71317]:   </devices>
Apr 14 04:31:12 user nova-compute[71317]: </domain>
Apr 14 04:31:12 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:31:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-496557296',display_name='tempest-ServerActionsTestJSON-server-496557296',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-496557296',id=34,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-olssq35z',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:31:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:69:b3:83,bridge_name='br-int',has_traffic_filtering=True,id=bc83f476-5176-4301-8272-80034f4a0daf,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbc83f476-51') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG os_vif [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:69:b3:83,bridge_name='br-int',has_traffic_filtering=True,id=bc83f476-5176-4301-8272-80034f4a0daf,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbc83f476-51') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapbc83f476-51, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapbc83f476-51, col_values=(('external_ids', {'iface-id': 'bc83f476-5176-4301-8272-80034f4a0daf', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:69:b3:83', 'vm-uuid': '123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:12 user nova-compute[71317]: INFO os_vif [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:69:b3:83,bridge_name='br-int',has_traffic_filtering=True,id=bc83f476-5176-4301-8272-80034f4a0daf,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbc83f476-51')
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:31:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No VIF found with MAC fa:16:3e:69:b3:83, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:31:13 user nova-compute[71317]: DEBUG nova.network.neutron [req-25761e43-cb69-4c40-bc44-d67644af0713 req-7fd7bb17-8cbb-453c-b460-0e2d91340c7e service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Updated VIF entry in instance network info cache for port bc83f476-5176-4301-8272-80034f4a0daf. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:31:13 user nova-compute[71317]: DEBUG nova.network.neutron [req-25761e43-cb69-4c40-bc44-d67644af0713 req-7fd7bb17-8cbb-453c-b460-0e2d91340c7e service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Updating instance_info_cache with network_info: [{"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:31:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-25761e43-cb69-4c40-bc44-d67644af0713 req-7fd7bb17-8cbb-453c-b460-0e2d91340c7e service nova] Releasing lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:31:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-160bd61e-1d3c-42ff-bac2-2cf95dc60ca6 req-6d9c9b77-cbf0-4e23-a1ca-fbb45fc5eb83 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Received event network-vif-plugged-bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-160bd61e-1d3c-42ff-bac2-2cf95dc60ca6 req-6d9c9b77-cbf0-4e23-a1ca-fbb45fc5eb83 service nova] Acquiring lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-160bd61e-1d3c-42ff-bac2-2cf95dc60ca6 req-6d9c9b77-cbf0-4e23-a1ca-fbb45fc5eb83 service nova] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-160bd61e-1d3c-42ff-bac2-2cf95dc60ca6 req-6d9c9b77-cbf0-4e23-a1ca-fbb45fc5eb83 service nova] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-160bd61e-1d3c-42ff-bac2-2cf95dc60ca6 req-6d9c9b77-cbf0-4e23-a1ca-fbb45fc5eb83 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] No waiting events found dispatching network-vif-plugged-bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:31:14 user nova-compute[71317]: WARNING nova.compute.manager [req-160bd61e-1d3c-42ff-bac2-2cf95dc60ca6 req-6d9c9b77-cbf0-4e23-a1ca-fbb45fc5eb83 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Received unexpected event network-vif-plugged-bc83f476-5176-4301-8272-80034f4a0daf for instance with vm_state building and task_state spawning.
Apr 14 04:31:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446676.3340428, 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:31:16 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] VM Resumed (Lifecycle Event)
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:31:16 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Instance spawned successfully.
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.compute.manager [req-1e94a9f6-45f6-416a-b819-da953f72c09e req-f6ffa2f2-07be-43cd-a76e-002cad2a0bb5 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Received event network-vif-plugged-bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1e94a9f6-45f6-416a-b819-da953f72c09e req-f6ffa2f2-07be-43cd-a76e-002cad2a0bb5 service nova] Acquiring lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1e94a9f6-45f6-416a-b819-da953f72c09e req-f6ffa2f2-07be-43cd-a76e-002cad2a0bb5 service nova] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1e94a9f6-45f6-416a-b819-da953f72c09e req-f6ffa2f2-07be-43cd-a76e-002cad2a0bb5 service nova] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.compute.manager [req-1e94a9f6-45f6-416a-b819-da953f72c09e req-f6ffa2f2-07be-43cd-a76e-002cad2a0bb5 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] No waiting events found dispatching network-vif-plugged-bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:31:16 user nova-compute[71317]: WARNING nova.compute.manager [req-1e94a9f6-45f6-416a-b819-da953f72c09e req-f6ffa2f2-07be-43cd-a76e-002cad2a0bb5 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Received unexpected event network-vif-plugged-bc83f476-5176-4301-8272-80034f4a0daf for instance with vm_state building and task_state spawning.
Apr 14 04:31:16 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446676.3414054, 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:31:16 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] VM Started (Lifecycle Event)
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:31:16 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:31:16 user nova-compute[71317]: INFO nova.compute.manager [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Took 6.13 seconds to spawn the instance on the hypervisor.
Apr 14 04:31:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:31:16 user nova-compute[71317]: INFO nova.compute.manager [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Took 7.00 seconds to build instance.
Apr 14 04:31:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bebd9aa3-87e4-411e-af3c-ad0fe137f50e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.098s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:34 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:31:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:31:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:31:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:31:34 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:31:35 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updating instance_info_cache with network_info: [{"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:31:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-cab84e5c-99ba-4e61-bf68-0a02b31ded5c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:31:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:31:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:31:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:31:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:31:36 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:31:36 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:31:37 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:31:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:31:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.148s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:31:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:40 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:31:40 user nova-compute[71317]: INFO nova.compute.manager [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] instance snapshotting
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.212s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:41 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Beginning live snapshot process
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.192s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json -f qcow2 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json -f qcow2" returned: 0 in 0.175s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json -f qcow2 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json -f qcow2" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpcgo59lbx/9c5bee1de5c14011b7e8db2991ae2554.delta 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpcgo59lbx/9c5bee1de5c14011b7e8db2991ae2554.delta 1073741824" returned: 0 in 0.057s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:41 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Quiescing instance not available: QEMU guest agent is not enabled.
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] COPY block job progress, current cursor: 43778048 final cursor: 43778048 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:31:43 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:31:43 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7633MB free_disk=26.283477783203125GB free_vcpus=1 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:43 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Skipping quiescing instance: QEMU guest agent is not enabled.
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpcgo59lbx/9c5bee1de5c14011b7e8db2991ae2554.delta /opt/stack/data/nova/instances/snapshots/tmpcgo59lbx/9c5bee1de5c14011b7e8db2991ae2554 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance af7befec-3f77-4eb7-be53-d1b1e791713e actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 19a166de-f0c9-43f2-8c88-585e0d5d22cc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 390ca845-5660-49d0-bd8a-fcfc39176ca5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance ca085872-69ff-41e4-be13-ffaa9a7cca82 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1920MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpcgo59lbx/9c5bee1de5c14011b7e8db2991ae2554.delta /opt/stack/data/nova/instances/snapshots/tmpcgo59lbx/9c5bee1de5c14011b7e8db2991ae2554" returned: 0 in 0.327s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:43 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Snapshot extracted, beginning image upload
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:31:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.635s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:44 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:31:46 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Snapshot image upload complete
Apr 14 04:31:46 user nova-compute[71317]: INFO nova.compute.manager [None req-5aad4665-ec1e-4238-a3a4-3c6604bc67cc tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Took 5.09 seconds to snapshot the instance on the hypervisor.
Apr 14 04:31:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:49 user nova-compute[71317]: INFO nova.compute.manager [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Terminating instance
Apr 14 04:31:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-0ee389a6-7de4-4e3d-9ced-c4b3026c9625 req-fef296fe-19b4-4811-bce5-19882f3bb3d1 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-unplugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0ee389a6-7de4-4e3d-9ced-c4b3026c9625 req-fef296fe-19b4-4811-bce5-19882f3bb3d1 service nova] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0ee389a6-7de4-4e3d-9ced-c4b3026c9625 req-fef296fe-19b4-4811-bce5-19882f3bb3d1 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0ee389a6-7de4-4e3d-9ced-c4b3026c9625 req-fef296fe-19b4-4811-bce5-19882f3bb3d1 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-0ee389a6-7de4-4e3d-9ced-c4b3026c9625 req-fef296fe-19b4-4811-bce5-19882f3bb3d1 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] No waiting events found dispatching network-vif-unplugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:31:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-0ee389a6-7de4-4e3d-9ced-c4b3026c9625 req-fef296fe-19b4-4811-bce5-19882f3bb3d1 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-unplugged-e3868733-d538-4469-9a3e-c02014ec9066 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:50 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Instance destroyed successfully.
Apr 14 04:31:50 user nova-compute[71317]: DEBUG nova.objects.instance [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'resources' on Instance uuid cab84e5c-99ba-4e61-bf68-0a02b31ded5c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:25:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1081038034',display_name='tempest-ServerRescueNegativeTestJSON-server-1081038034',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1081038034',id=25,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCb+23dydPc8XvPesmHXUsmKMTGTpSa5GCZHGQJYeeij6pIqTWqaFg9yzcA/VhrLxaLND3aQCfGfHGfvo53ucWJsjWgHFiCEptESTMFy12g7udsn7vnD6U5hRBURzXxHFg==',key_name='tempest-keypair-1320453961',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:27:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='33320dc2f2c14ecfa5abf6574029c36c',ramdisk_id='',reservation_id='r-ym9z46z3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-508041341',owner_user_name='tempest-ServerRescueNegativeTestJSON-508041341-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:29:13Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5234914aec8e443b8a38ac6b38729031',uuid=cab84e5c-99ba-4e61-bf68-0a02b31ded5c,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converting VIF {"id": "e3868733-d538-4469-9a3e-c02014ec9066", "address": "fa:16:3e:71:08:35", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3868733-d5", "ovs_interfaceid": "e3868733-d538-4469-9a3e-c02014ec9066", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:71:08:35,bridge_name='br-int',has_traffic_filtering=True,id=e3868733-d538-4469-9a3e-c02014ec9066,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3868733-d5') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG os_vif [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:71:08:35,bridge_name='br-int',has_traffic_filtering=True,id=e3868733-d538-4469-9a3e-c02014ec9066,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3868733-d5') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape3868733-d5, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:31:50 user nova-compute[71317]: INFO os_vif [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:71:08:35,bridge_name='br-int',has_traffic_filtering=True,id=e3868733-d538-4469-9a3e-c02014ec9066,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3868733-d5')
Apr 14 04:31:50 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Deleting instance files /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c_del
Apr 14 04:31:50 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Deletion of /opt/stack/data/nova/instances/cab84e5c-99ba-4e61-bf68-0a02b31ded5c_del complete
Apr 14 04:31:50 user nova-compute[71317]: INFO nova.compute.manager [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Took 0.91 seconds to destroy the instance on the hypervisor.
Apr 14 04:31:50 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:31:50 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:31:51 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:31:51 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Took 0.92 seconds to deallocate network for instance.
Apr 14 04:31:51 user nova-compute[71317]: DEBUG nova.compute.manager [req-a47af9b4-966f-41e0-a006-5af5ae324937 req-9f32bbfe-43d5-43ad-b327-243b2250f487 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-deleted-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:51 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:31:51 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:31:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.389s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:51 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Deleted allocations for instance cab84e5c-99ba-4e61-bf68-0a02b31ded5c
Apr 14 04:31:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4e3655cf-0dc2-437c-97d2-06802994d8e8 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.422s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-36552ad5-f688-470e-8c47-f88f45d9297e req-fd02f248-fcc9-4194-b359-a4f3941ac4b5 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36552ad5-f688-470e-8c47-f88f45d9297e req-fd02f248-fcc9-4194-b359-a4f3941ac4b5 service nova] Acquiring lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36552ad5-f688-470e-8c47-f88f45d9297e req-fd02f248-fcc9-4194-b359-a4f3941ac4b5 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36552ad5-f688-470e-8c47-f88f45d9297e req-fd02f248-fcc9-4194-b359-a4f3941ac4b5 service nova] Lock "cab84e5c-99ba-4e61-bf68-0a02b31ded5c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-36552ad5-f688-470e-8c47-f88f45d9297e req-fd02f248-fcc9-4194-b359-a4f3941ac4b5 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] No waiting events found dispatching network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:31:52 user nova-compute[71317]: WARNING nova.compute.manager [req-36552ad5-f688-470e-8c47-f88f45d9297e req-fd02f248-fcc9-4194-b359-a4f3941ac4b5 service nova] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Received unexpected event network-vif-plugged-e3868733-d538-4469-9a3e-c02014ec9066 for instance with vm_state deleted and task_state None.
Apr 14 04:31:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-f2d14f0d-50aa-473d-919f-180a9b91b9b2 req-cebdc234-eb17-47ae-9d86-419e1328a794 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-changed-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-f2d14f0d-50aa-473d-919f-180a9b91b9b2 req-cebdc234-eb17-47ae-9d86-419e1328a794 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Refreshing instance network info cache due to event network-changed-61ec63cd-2733-40c4-a8c6-9243a6a3c144. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f2d14f0d-50aa-473d-919f-180a9b91b9b2 req-cebdc234-eb17-47ae-9d86-419e1328a794 service nova] Acquiring lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f2d14f0d-50aa-473d-919f-180a9b91b9b2 req-cebdc234-eb17-47ae-9d86-419e1328a794 service nova] Acquired lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG nova.network.neutron [req-f2d14f0d-50aa-473d-919f-180a9b91b9b2 req-cebdc234-eb17-47ae-9d86-419e1328a794 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Refreshing network info cache for port 61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:53 user nova-compute[71317]: INFO nova.compute.manager [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Terminating instance
Apr 14 04:31:53 user nova-compute[71317]: DEBUG nova.compute.manager [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-f2d14f0d-50aa-473d-919f-180a9b91b9b2 req-cebdc234-eb17-47ae-9d86-419e1328a794 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updated VIF entry in instance network info cache for port 61ec63cd-2733-40c4-a8c6-9243a6a3c144. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-f2d14f0d-50aa-473d-919f-180a9b91b9b2 req-cebdc234-eb17-47ae-9d86-419e1328a794 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updating instance_info_cache with network_info: [{"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f2d14f0d-50aa-473d-919f-180a9b91b9b2 req-cebdc234-eb17-47ae-9d86-419e1328a794 service nova] Releasing lock "refresh_cache-6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:31:54 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Instance destroyed successfully.
Apr 14 04:31:54 user nova-compute[71317]: DEBUG nova.objects.instance [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lazy-loading 'resources' on Instance uuid 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:26:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-2065386753',display_name='tempest-TestShelveInstance-server-2065386753',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-2065386753',id=28,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGGk9hQ8wpWxQD87wrkXLIqSgBhES95isERnHNc/wFOMwogmDvk0s+HHEuuYbkzB2pEyv6u9qIhQ2bQkxjPV0WSr048nFyVVwxTOQQiEcmHX8pNahIhzR1Wk0Uv6MFb63A==',key_name='tempest-TestShelveInstance-2060150193',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:30:20Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='b0e15d3d3e0943c4b6bd54abf2de5fee',ramdisk_id='',reservation_id='r-osj4jtne',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1458140851',owner_user_name='tempest-TestShelveInstance-1458140851-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:30:21Z,user_data=None,user_id='36b6981092424de781e8403946f956d3',uuid=6a510dd3-bb77-4f6d-bdfd-180ce1d31c71,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Converting VIF {"id": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "address": "fa:16:3e:fb:2e:cd", "network": {"id": "f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9", "bridge": "br-int", "label": "tempest-TestShelveInstance-1668366353-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.166", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b0e15d3d3e0943c4b6bd54abf2de5fee", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap61ec63cd-27", "ovs_interfaceid": "61ec63cd-2733-40c4-a8c6-9243a6a3c144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_vif [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap61ec63cd-27, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:54 user nova-compute[71317]: INFO os_vif [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:2e:cd,bridge_name='br-int',has_traffic_filtering=True,id=61ec63cd-2733-40c4-a8c6-9243a6a3c144,network=Network(f844ce92-c1a2-47aa-8c67-fe7fb4dbacc9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap61ec63cd-27')
Apr 14 04:31:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pRhHtNJB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:31:54 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6735e96d-30e4-4651-bb9a-e8659b5e3145]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:31:54 user nova-compute[71317]: No targets found.
Apr 14 04:31:54 user nova-compute[71317]: iSNS:
Apr 14 04:31:54 user nova-compute[71317]: No targets found.
Apr 14 04:31:54 user nova-compute[71317]: STATIC:
Apr 14 04:31:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb
Apr 14 04:31:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:31:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:31:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895
Apr 14 04:31:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:31:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:31:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:31:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:31:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:31:54 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4
Apr 14 04:31:54 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:31:54 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:31:54 user nova-compute[71317]: FIRMWARE:
Apr 14 04:31:54 user nova-compute[71317]: No targets found.
Apr 14 04:31:54 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:31:54 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:31:54 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bffb98f4-3ded-41e2-a291-335b6926a2ee]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c037e94f-11b6-4884-97b9-6917a6db5290]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:31:54 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:31:54 user nova-compute[71317]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:31:54 user nova-compute[71317]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)
Apr 14 04:31:54 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:31:54 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:31:54 user nova-compute[71317]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 (non-flash)
Apr 14 04:31:54 user nova-compute[71317]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)
Apr 14 04:31:54 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c41fc550>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a32dd6f-850f-44b5-aa4e-d0b12f51b104]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d026aae-05c4-4852-b335-e21d2d7d1934]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a64dfe33-0557-4663-8d45-068219c287ea]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.055s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[91384224-d93c-4a24-ae7c-062e973ff413]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[20c9bd04-14f2-451c-857e-c6afa8037eab]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b7752e2-17ec-4f0e-90ad-a68ca8cd3a96]: (4, ('Logging out of session [sid: 41, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260]\nLogout of [sid: 41, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 41, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260]
Apr 14 04:31:54 user nova-compute[71317]: Logout of [sid: 41, target: iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4, portal: 172.16.0.220,3260] successful.
Apr 14 04:31:54 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bcb3b184-a10d-482d-8c49-1bee42873be4 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2dade6f5-e513-4429-aeca-35b6c1a1f19a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.342s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] <== disconnect_volume: return (343ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:31:54 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Deleting instance files /opt/stack/data/nova/instances/6a510dd3-bb77-4f6d-bdfd-180ce1d31c71_del
Apr 14 04:31:54 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Deletion of /opt/stack/data/nova/instances/6a510dd3-bb77-4f6d-bdfd-180ce1d31c71_del complete
Apr 14 04:31:54 user nova-compute[71317]: INFO nova.compute.manager [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Took 1.26 seconds to destroy the instance on the hypervisor.
Apr 14 04:31:54 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:31:54 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:31:55 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:31:55 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Took 0.65 seconds to deallocate network for instance.
Apr 14 04:31:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-cbaf2caf-0331-4dba-bd4d-e411c16d9c1a req-0c996982-098d-4018-9f58-1072c678d6b4 service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-vif-deleted-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-cdb17712-4845-4cbd-b0c5-2b8976deafc3 req-abfa80d6-7bea-4485-92a5-017e028e9c1f service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-vif-unplugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cdb17712-4845-4cbd-b0c5-2b8976deafc3 req-abfa80d6-7bea-4485-92a5-017e028e9c1f service nova] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cdb17712-4845-4cbd-b0c5-2b8976deafc3 req-abfa80d6-7bea-4485-92a5-017e028e9c1f service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cdb17712-4845-4cbd-b0c5-2b8976deafc3 req-abfa80d6-7bea-4485-92a5-017e028e9c1f service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-cdb17712-4845-4cbd-b0c5-2b8976deafc3 req-abfa80d6-7bea-4485-92a5-017e028e9c1f service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] No waiting events found dispatching network-vif-unplugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:31:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-cdb17712-4845-4cbd-b0c5-2b8976deafc3 req-abfa80d6-7bea-4485-92a5-017e028e9c1f service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-vif-unplugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:31:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-cdb17712-4845-4cbd-b0c5-2b8976deafc3 req-abfa80d6-7bea-4485-92a5-017e028e9c1f service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received event network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:31:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cdb17712-4845-4cbd-b0c5-2b8976deafc3 req-abfa80d6-7bea-4485-92a5-017e028e9c1f service nova] Acquiring lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cdb17712-4845-4cbd-b0c5-2b8976deafc3 req-abfa80d6-7bea-4485-92a5-017e028e9c1f service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cdb17712-4845-4cbd-b0c5-2b8976deafc3 req-abfa80d6-7bea-4485-92a5-017e028e9c1f service nova] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-cdb17712-4845-4cbd-b0c5-2b8976deafc3 req-abfa80d6-7bea-4485-92a5-017e028e9c1f service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] No waiting events found dispatching network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:31:55 user nova-compute[71317]: WARNING nova.compute.manager [req-cdb17712-4845-4cbd-b0c5-2b8976deafc3 req-abfa80d6-7bea-4485-92a5-017e028e9c1f service nova] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Received unexpected event network-vif-plugged-61ec63cd-2733-40c4-a8c6-9243a6a3c144 for instance with vm_state active and task_state deleting.
Apr 14 04:31:58 user nova-compute[71317]: INFO nova.compute.manager [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Took 2.75 seconds to detach 1 volumes for instance.
Apr 14 04:31:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Deleting volume: bcb3b184-a10d-482d-8c49-1bee42873be4 {{(pid=71317) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 14 04:31:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:31:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:31:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:31:58 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:31:58 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:31:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.331s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:58 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Deleted allocations for instance 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71
Apr 14 04:31:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-33a6dd15-0f49-4e3b-91a5-dbe3b02439ba tempest-TestShelveInstance-1458140851 tempest-TestShelveInstance-1458140851-project-member] Lock "6a510dd3-bb77-4f6d-bdfd-180ce1d31c71" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 5.255s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:31:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:05 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446710.4368865, cab84e5c-99ba-4e61-bf68-0a02b31ded5c => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:32:05 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] VM Stopped (Lifecycle Event)
Apr 14 04:32:05 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d15adc1b-6b73-4b70-9240-9d90aa1e4247 None None] [instance: cab84e5c-99ba-4e61-bf68-0a02b31ded5c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-891ec3a3-76ab-4cef-849e-43326a7aa983 req-8c049063-b60e-450b-bca0-04bf5dbccbcc service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Received event network-changed-488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-891ec3a3-76ab-4cef-849e-43326a7aa983 req-8c049063-b60e-450b-bca0-04bf5dbccbcc service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Refreshing instance network info cache due to event network-changed-488d7017-4718-4554-80f0-7fb119efc417. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-891ec3a3-76ab-4cef-849e-43326a7aa983 req-8c049063-b60e-450b-bca0-04bf5dbccbcc service nova] Acquiring lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-891ec3a3-76ab-4cef-849e-43326a7aa983 req-8c049063-b60e-450b-bca0-04bf5dbccbcc service nova] Acquired lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG nova.network.neutron [req-891ec3a3-76ab-4cef-849e-43326a7aa983 req-8c049063-b60e-450b-bca0-04bf5dbccbcc service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Refreshing network info cache for port 488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lazy-loading 'flavor' on Instance uuid 390ca845-5660-49d0-bd8a-fcfc39176ca5 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.057s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:06 user nova-compute[71317]: INFO nova.compute.manager [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Attaching volume cd931337-f058-4d2e-bb9b-21d50e96a28b to /dev/vdb
Apr 14 04:32:06 user nova-compute[71317]: DEBUG os_brick.utils [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ac5c807-8771-45d4-9371-f9c9ec90443a]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[561551e8-558b-48e2-9806-4cc0f7daaa27]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d419c3b6-0fa5-4f5e-a660-abadf6fc8941]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4fbe447d-5a05-46f1-a3d7-1c43d28bacf9]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG nova.network.neutron [req-891ec3a3-76ab-4cef-849e-43326a7aa983 req-8c049063-b60e-450b-bca0-04bf5dbccbcc service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Updated VIF entry in instance network info cache for port 488d7017-4718-4554-80f0-7fb119efc417. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG nova.network.neutron [req-891ec3a3-76ab-4cef-849e-43326a7aa983 req-8c049063-b60e-450b-bca0-04bf5dbccbcc service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Updating instance_info_cache with network_info: [{"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.112", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:32:06 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[37a948d2-1de6-4c3b-a7dd-7d0a8140556e]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG os_brick.utils [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] <== get_connector_properties: return (142ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Updating existing volume attachment record: 0b5e165e-40ef-4330-a65d-5a6b0b96363f {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:32:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-891ec3a3-76ab-4cef-849e-43326a7aa983 req-8c049063-b60e-450b-bca0-04bf5dbccbcc service nova] Releasing lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:32:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:09 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446714.2866719, 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:32:09 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] VM Stopped (Lifecycle Event)
Apr 14 04:32:09 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4302c213-3bb3-430a-95c5-df62b2b9c752 None None] [instance: 6a510dd3-bb77-4f6d-bdfd-180ce1d31c71] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:32:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KfcZYljW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:32:13 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:32:13 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:32:13 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5ef1574-86f1-4105-9df7-0c0060f0dbfd]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:32:13 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8c9c4e1-db13-40f9-bdfb-a92675d7fb0d]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b] added
Apr 14 04:32:13 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba5f4197-2d4a-4b24-899f-597df65988e9]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ff31bbea-95cb-4bad-8c6c-3d9022fcc9a1]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v KfcZYljW {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v KfcZYljW" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5be8d8b-e90b-4bb6-9480-e34f61feb9f9]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'KfcZYljW'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab988d85-592b-42b0-8149-5d06a9b6e28b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[12c5864b-9b7c-4b23-b93d-7f19f9caf4f7]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:32:13 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:32:13 user nova-compute[71317]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)
Apr 14 04:32:13 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:32:13 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:32:13 user nova-compute[71317]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)
Apr 14 04:32:13 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:32:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --login" returned: 0 in 0.061s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a32dbae-6b6e-4e5d-a502-1a4a1b67776e]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b, portal: 172.16.0.220,3260]
Apr 14 04:32:14 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b, portal: 172.16.0.220,3260] successful.
Apr 14 04:32:14 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e4447fd-b00d-4aeb-8182-5ca9d097e2ed]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f957918-19b5-4c72-89e1-266245d37488]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:32:14 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:32:14 user nova-compute[71317]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)
Apr 14 04:32:14 user nova-compute[71317]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b (non-flash)
Apr 14 04:32:14 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:32:14 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:32:14 user nova-compute[71317]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)
Apr 14 04:32:14 user nova-compute[71317]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b (non-flash)
Apr 14 04:32:14 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.290s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] HCTL ('34', '-', '-', 0) found on session 43 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4253fedf-eedf-4122-b545-1789aa416e47]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Searching for a device in session 43 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KfcZYljW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:32:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.328s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:32:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] <== connect_volume: return (1330ms) {'type': 'block', 'scsi_wwn': '23262646166356330', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:32:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23262646166356330', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:32:15 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lazy-loading 'flavor' on Instance uuid 390ca845-5660-49d0-bd8a-fcfc39176ca5 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:32:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:32:15 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:32:15 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:32:15 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:32:15 user nova-compute[71317]:   <serial>cd931337-f058-4d2e-bb9b-21d50e96a28b</serial>
Apr 14 04:32:15 user nova-compute[71317]: </disk>
Apr 14 04:32:15 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:32:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:32:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:32:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] No VIF found with MAC fa:16:3e:2f:d9:94, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:32:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d0850cc3-8eee-447c-8f6e-79920a1b6d34 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.234s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:17 user nova-compute[71317]: INFO nova.compute.manager [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Detaching volume cd931337-f058-4d2e-bb9b-21d50e96a28b
Apr 14 04:32:17 user nova-compute[71317]: INFO nova.virt.block_device [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Attempting to driver detach volume cd931337-f058-4d2e-bb9b-21d50e96a28b from mountpoint /dev/vdb
Apr 14 04:32:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Attempting to detach device vdb from instance 390ca845-5660-49d0-bd8a-fcfc39176ca5 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:32:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:32:17 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:32:17 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:32:17 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:32:17 user nova-compute[71317]:   <serial>cd931337-f058-4d2e-bb9b-21d50e96a28b</serial>
Apr 14 04:32:17 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:32:17 user nova-compute[71317]: </disk>
Apr 14 04:32:17 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:32:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Successfully detached device vdb from instance 390ca845-5660-49d0-bd8a-fcfc39176ca5 from the persistent domain config.
Apr 14 04:32:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 390ca845-5660-49d0-bd8a-fcfc39176ca5 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:32:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:32:17 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:32:17 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:32:17 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:32:17 user nova-compute[71317]:   <serial>cd931337-f058-4d2e-bb9b-21d50e96a28b</serial>
Apr 14 04:32:17 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:32:17 user nova-compute[71317]: </disk>
Apr 14 04:32:17 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446738.22094, 390ca845-5660-49d0-bd8a-fcfc39176ca5 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 390ca845-5660-49d0-bd8a-fcfc39176ca5 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:32:18 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Successfully detached device vdb from instance 390ca845-5660-49d0-bd8a-fcfc39176ca5 from the live domain config.
Apr 14 04:32:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KfcZYljW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:32:18 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac0b7900-a710-407c-86d3-4d119ffd5cab]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:32:18 user nova-compute[71317]: No targets found.
Apr 14 04:32:18 user nova-compute[71317]: iSNS:
Apr 14 04:32:18 user nova-compute[71317]: No targets found.
Apr 14 04:32:18 user nova-compute[71317]: STATIC:
Apr 14 04:32:18 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b
Apr 14 04:32:18 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:32:18 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:32:18 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb
Apr 14 04:32:18 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:32:18 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:32:18 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895
Apr 14 04:32:18 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:32:18 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:32:18 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:32:18 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:32:18 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:32:18 user nova-compute[71317]: FIRMWARE:
Apr 14 04:32:18 user nova-compute[71317]: No targets found.
Apr 14 04:32:18 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:32:18 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:32:18 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[03d94624-9fb7-4ccf-97e5-f74688fc5a9c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7ce9d2f-0092-41dc-958e-1de76c02dcc6]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:32:18 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:32:18 user nova-compute[71317]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)
Apr 14 04:32:18 user nova-compute[71317]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b (non-flash)
Apr 14 04:32:18 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:32:18 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:32:18 user nova-compute[71317]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)
Apr 14 04:32:18 user nova-compute[71317]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b (non-flash)
Apr 14 04:32:18 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8490214310>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f3b3469-21af-409d-b473-735222f55ad9]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[35aef93e-9650-4b2b-aae8-440dc4793a1f]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cbc366c1-8875-49b1-a2bd-04be0cebf7cd]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.041s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6af17f07-06a7-4919-bfc9-40368a8accae]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb6bdfb2-deec-45e2-94ba-c9481cca2f2e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --logout" returned: 0 in 0.059s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef44aea4-0fe7-49f0-a3f1-f96a7dec899d]: (4, ('Logging out of session [sid: 43, target: iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b, portal: 172.16.0.220,3260]\nLogout of [sid: 43, target: iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 43, target: iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b, portal: 172.16.0.220,3260]
Apr 14 04:32:18 user nova-compute[71317]: Logout of [sid: 43, target: iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b, portal: 172.16.0.220,3260] successful.
Apr 14 04:32:18 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cd931337-f058-4d2e-bb9b-21d50e96a28b -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[77d03b0b-5081-4862-b1eb-5587bba6bb56]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.309s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] <== disconnect_volume: return (311ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:32:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:32:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:23 user nova-compute[71317]: DEBUG nova.objects.instance [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lazy-loading 'flavor' on Instance uuid 390ca845-5660-49d0-bd8a-fcfc39176ca5 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:32:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-f486042f-23ac-4de7-afce-a9f775a4ebe7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 6.055s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:30 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:31 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:31 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:33 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:33 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:32:33 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:32:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:32:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:32:33 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:32:33 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lazy-loading 'info_cache' on Instance uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:32:33 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Updating instance_info_cache with network_info: [{"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:32:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:32:33 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:32:34 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:32:37 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:38 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:39 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:32:39 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 04:32:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] There are 0 instances to clean {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 04:32:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk.rescue --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "ca085872-69ff-41e4-be13-ffaa9a7cca82" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:41 user nova-compute[71317]: INFO nova.compute.manager [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Terminating instance
Apr 14 04:32:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-0f9287f1-daa6-4737-aa40-e41303e77a30 req-b3ae44a7-1a71-4f65-be0b-b8ceef4356a9 service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Received event network-vif-unplugged-f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0f9287f1-daa6-4737-aa40-e41303e77a30 req-b3ae44a7-1a71-4f65-be0b-b8ceef4356a9 service nova] Acquiring lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0f9287f1-daa6-4737-aa40-e41303e77a30 req-b3ae44a7-1a71-4f65-be0b-b8ceef4356a9 service nova] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0f9287f1-daa6-4737-aa40-e41303e77a30 req-b3ae44a7-1a71-4f65-be0b-b8ceef4356a9 service nova] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-0f9287f1-daa6-4737-aa40-e41303e77a30 req-b3ae44a7-1a71-4f65-be0b-b8ceef4356a9 service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] No waiting events found dispatching network-vif-unplugged-f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-0f9287f1-daa6-4737-aa40-e41303e77a30 req-b3ae44a7-1a71-4f65-be0b-b8ceef4356a9 service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Received event network-vif-unplugged-f010668b-3ce4-4f88-a116-dd9a67663985 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:41 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Instance destroyed successfully.
Apr 14 04:32:41 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lazy-loading 'resources' on Instance uuid ca085872-69ff-41e4-be13-ffaa9a7cca82 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:30:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-453223105',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-453223105',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-453223105',id=33,image_ref='37c9a7e6-d203-4290-ba7e-602fdff12012',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:31:07Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-7vc9kziz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='member,reader',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-1780160917',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member',image_root_device_name='/dev/vda',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:31:07Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=ca085872-69ff-41e4-be13-ffaa9a7cca82,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f010668b-3ce4-4f88-a116-dd9a67663985", "address": "fa:16:3e:df:f8:42", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf010668b-3c", "ovs_interfaceid": "f010668b-3ce4-4f88-a116-dd9a67663985", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "f010668b-3ce4-4f88-a116-dd9a67663985", "address": "fa:16:3e:df:f8:42", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf010668b-3c", "ovs_interfaceid": "f010668b-3ce4-4f88-a116-dd9a67663985", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:df:f8:42,bridge_name='br-int',has_traffic_filtering=True,id=f010668b-3ce4-4f88-a116-dd9a67663985,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf010668b-3c') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG os_vif [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:df:f8:42,bridge_name='br-int',has_traffic_filtering=True,id=f010668b-3ce4-4f88-a116-dd9a67663985,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf010668b-3c') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapf010668b-3c, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:32:41 user nova-compute[71317]: INFO os_vif [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:df:f8:42,bridge_name='br-int',has_traffic_filtering=True,id=f010668b-3ce4-4f88-a116-dd9a67663985,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf010668b-3c')
Apr 14 04:32:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qBXmfngt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:32:41 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:32:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e8de5c9-1d80-4931-8700-72b789d9037f]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:32:41 user nova-compute[71317]: No targets found.
Apr 14 04:32:41 user nova-compute[71317]: iSNS:
Apr 14 04:32:41 user nova-compute[71317]: No targets found.
Apr 14 04:32:41 user nova-compute[71317]: STATIC:
Apr 14 04:32:41 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb
Apr 14 04:32:41 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:32:41 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:32:41 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895
Apr 14 04:32:41 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:32:41 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:32:41 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:32:41 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:32:41 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:32:41 user nova-compute[71317]: FIRMWARE:
Apr 14 04:32:41 user nova-compute[71317]: No targets found.
Apr 14 04:32:41 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:32:41 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:32:41 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3010b348-88fc-4cac-be3d-621d5ca2f472]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[95181eb8-e167-4278-8512-fa434efc23bc]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:32:41 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:32:41 user nova-compute[71317]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)
Apr 14 04:32:41 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:32:41 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:32:41 user nova-compute[71317]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb (non-flash)
Apr 14 04:32:41 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84907ca320>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb'): ({'sdd'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Removing single pathed devices sdd {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:32:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[33f63434-94e7-4093-b5ac-a3abee64171f]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5dce4088-52ff-4cdf-8710-9d0bf08c3884]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flushing IO for device /dev/sdd {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e517e9b-5d13-49cd-8f1b-12ba8b553fdd]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.063s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d57ab210-7e9f-419a-9d2f-89a686c4e59d]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1c95adf-eec0-45a3-9851-e3f1e655c34d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc3ada90-37e1-4366-88f6-bb9225336be4]: (4, ('Logging out of session [sid: 42, target: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb, portal: 172.16.0.220,3260]\nLogout of [sid: 42, target: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 42, target: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb, portal: 172.16.0.220,3260]
Apr 14 04:32:42 user nova-compute[71317]: Logout of [sid: 42, target: iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb, portal: 172.16.0.220,3260] successful.
Apr 14 04:32:42 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9e63107-a32f-4221-964d-9ee193d260fb -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1548bb3-7598-4993-a867-3e6949ec0289]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.435s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== disconnect_volume: return (437ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:32:42 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Deleting instance files /opt/stack/data/nova/instances/ca085872-69ff-41e4-be13-ffaa9a7cca82_del
Apr 14 04:32:42 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Deletion of /opt/stack/data/nova/instances/ca085872-69ff-41e4-be13-ffaa9a7cca82_del complete
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:42 user nova-compute[71317]: INFO nova.compute.manager [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Took 1.38 seconds to destroy the instance on the hypervisor.
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:43 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:32:43 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7885MB free_disk=26.263973236083984GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:32:43 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Took 0.95 seconds to deallocate network for instance.
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-99e560d8-1386-471f-a965-86cf81705835 req-5000da55-fce5-4afa-a058-5c47dc124480 service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Received event network-vif-deleted-f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance af7befec-3f77-4eb7-be53-d1b1e791713e actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 19a166de-f0c9-43f2-8c88-585e0d5d22cc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 390ca845-5660-49d0-bd8a-fcfc39176ca5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance ca085872-69ff-41e4-be13-ffaa9a7cca82 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing inventories for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating ProviderTree inventory for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating inventory in ProviderTree for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing aggregate associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, aggregates: None {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-073140f6-30fa-4b08-8614-21570d88cc76 req-b08f7523-16db-4bbc-8da5-4e3beac936fc service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Received event network-vif-plugged-f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-073140f6-30fa-4b08-8614-21570d88cc76 req-b08f7523-16db-4bbc-8da5-4e3beac936fc service nova] Acquiring lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-073140f6-30fa-4b08-8614-21570d88cc76 req-b08f7523-16db-4bbc-8da5-4e3beac936fc service nova] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-073140f6-30fa-4b08-8614-21570d88cc76 req-b08f7523-16db-4bbc-8da5-4e3beac936fc service nova] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-073140f6-30fa-4b08-8614-21570d88cc76 req-b08f7523-16db-4bbc-8da5-4e3beac936fc service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] No waiting events found dispatching network-vif-plugged-f010668b-3ce4-4f88-a116-dd9a67663985 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:32:43 user nova-compute[71317]: WARNING nova.compute.manager [req-073140f6-30fa-4b08-8614-21570d88cc76 req-b08f7523-16db-4bbc-8da5-4e3beac936fc service nova] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Received unexpected event network-vif-plugged-f010668b-3ce4-4f88-a116-dd9a67663985 for instance with vm_state active and task_state deleting.
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing trait associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, traits: COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSE41,HW_CPU_X86_SSE42,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_TRUSTED_CERTS,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_ACCELERATORS,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NET_VIF_MODEL_E1000,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_NODE,HW_CPU_X86_SSE2,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,HW_CPU_X86_SSSE3,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_STORAGE_BUS_USB,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_NE2K_PCI {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:32:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.784s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:44 user nova-compute[71317]: INFO nova.compute.manager [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Took 1.21 seconds to detach 1 volumes for instance.
Apr 14 04:32:44 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Deleting volume: e9e63107-a32f-4221-964d-9ee193d260fb {{(pid=71317) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 14 04:32:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:45 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:32:45 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:32:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.377s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:45 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Deleted allocations for instance ca085872-69ff-41e4-be13-ffaa9a7cca82
Apr 14 04:32:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a5bf3f94-86f7-4753-aa3d-ccd0f94df8e2 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "ca085872-69ff-41e4-be13-ffaa9a7cca82" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.259s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:45 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:45 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:46 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:32:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances with incomplete migration  {{(pid=71317) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 04:32:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:47 user nova-compute[71317]: INFO nova.compute.manager [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Terminating instance
Apr 14 04:32:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:32:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-4c7770dd-8bc0-46c1-9189-a2d279d32b82 req-2538c08a-235f-46d2-9d14-5a7518f10921 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received event network-vif-unplugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4c7770dd-8bc0-46c1-9189-a2d279d32b82 req-2538c08a-235f-46d2-9d14-5a7518f10921 service nova] Acquiring lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4c7770dd-8bc0-46c1-9189-a2d279d32b82 req-2538c08a-235f-46d2-9d14-5a7518f10921 service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4c7770dd-8bc0-46c1-9189-a2d279d32b82 req-2538c08a-235f-46d2-9d14-5a7518f10921 service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-4c7770dd-8bc0-46c1-9189-a2d279d32b82 req-2538c08a-235f-46d2-9d14-5a7518f10921 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] No waiting events found dispatching network-vif-unplugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-4c7770dd-8bc0-46c1-9189-a2d279d32b82 req-2538c08a-235f-46d2-9d14-5a7518f10921 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received event network-vif-unplugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:48 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Instance destroyed successfully.
Apr 14 04:32:48 user nova-compute[71317]: DEBUG nova.objects.instance [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'resources' on Instance uuid 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1602894262',display_name='tempest-ServerRescueNegativeTestJSON-server-1602894262',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1602894262',id=7,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:18:41Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='33320dc2f2c14ecfa5abf6574029c36c',ramdisk_id='',reservation_id='r-5xk8owm0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-508041341',owner_user_name='tempest-ServerRescueNegativeTestJSON-508041341-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:18:42Z,user_data=None,user_id='5234914aec8e443b8a38ac6b38729031',uuid=2f7e20cd-33ef-4f83-bcdb-a13b9b51370a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='rescued') vif={"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converting VIF {"id": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "address": "fa:16:3e:f5:07:c5", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1201ab18-d7", "ovs_interfaceid": "1201ab18-d7ef-4a92-a77e-d0ab69858538", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f5:07:c5,bridge_name='br-int',has_traffic_filtering=True,id=1201ab18-d7ef-4a92-a77e-d0ab69858538,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1201ab18-d7') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG os_vif [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:f5:07:c5,bridge_name='br-int',has_traffic_filtering=True,id=1201ab18-d7ef-4a92-a77e-d0ab69858538,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1201ab18-d7') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap1201ab18-d7, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:32:48 user nova-compute[71317]: INFO os_vif [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:f5:07:c5,bridge_name='br-int',has_traffic_filtering=True,id=1201ab18-d7ef-4a92-a77e-d0ab69858538,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1201ab18-d7')
Apr 14 04:32:48 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Deleting instance files /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a_del
Apr 14 04:32:48 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Deletion of /opt/stack/data/nova/instances/2f7e20cd-33ef-4f83-bcdb-a13b9b51370a_del complete
Apr 14 04:32:48 user nova-compute[71317]: INFO nova.compute.manager [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Took 0.95 seconds to destroy the instance on the hypervisor.
Apr 14 04:32:48 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "970e81d2-c227-4eab-92dd-1ed732e67968" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:32:49 user nova-compute[71317]: INFO nova.compute.claims [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Claim successful on node user
Apr 14 04:32:49 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Took 0.64 seconds to deallocate network for instance.
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.compute.manager [req-befd56f6-b354-49d1-b42e-8b5765aa1123 req-a7d84113-db93-4d58-a632-4e81cc39e2f5 service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received event network-vif-deleted-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.389s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.312s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.network.neutron [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:32:49 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:32:49 user nova-compute[71317]: INFO nova.virt.block_device [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Booting with volume-backed-image 5c55a1da-f6f8-4442-939b-f0cb72ea4327 at /dev/vda
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.policy [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '6ec7103e06e643a6b9b2c908ba8d7b06', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd5995b934b0546c0983387e46e3b3df4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:32:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.322s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:49 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Deleted allocations for instance 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a
Apr 14 04:32:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57d65ad2-01be-4dc0-ac75-56d90ccf0e32 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.412s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:50 user nova-compute[71317]: DEBUG nova.compute.manager [req-99cfab93-4be6-40d8-b2bf-abe67eb86bb4 req-dac8ae25-866e-487d-a134-0963832e8cdf service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received event network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:32:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-99cfab93-4be6-40d8-b2bf-abe67eb86bb4 req-dac8ae25-866e-487d-a134-0963832e8cdf service nova] Acquiring lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:32:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-99cfab93-4be6-40d8-b2bf-abe67eb86bb4 req-dac8ae25-866e-487d-a134-0963832e8cdf service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:32:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-99cfab93-4be6-40d8-b2bf-abe67eb86bb4 req-dac8ae25-866e-487d-a134-0963832e8cdf service nova] Lock "2f7e20cd-33ef-4f83-bcdb-a13b9b51370a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:32:50 user nova-compute[71317]: DEBUG nova.compute.manager [req-99cfab93-4be6-40d8-b2bf-abe67eb86bb4 req-dac8ae25-866e-487d-a134-0963832e8cdf service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] No waiting events found dispatching network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:32:50 user nova-compute[71317]: WARNING nova.compute.manager [req-99cfab93-4be6-40d8-b2bf-abe67eb86bb4 req-dac8ae25-866e-487d-a134-0963832e8cdf service nova] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Received unexpected event network-vif-plugged-1201ab18-d7ef-4a92-a77e-d0ab69858538 for instance with vm_state deleted and task_state None.
Apr 14 04:32:51 user nova-compute[71317]: DEBUG nova.network.neutron [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Successfully created port: 45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:32:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Successfully updated port: 45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "refresh_cache-970e81d2-c227-4eab-92dd-1ed732e67968" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquired lock "refresh_cache-970e81d2-c227-4eab-92dd-1ed732e67968" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-ff31da45-c3aa-43fe-b52c-3c815dd7e7f4 req-21192af9-9d54-4393-91f5-dd46f7758b4c service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-changed-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-ff31da45-c3aa-43fe-b52c-3c815dd7e7f4 req-21192af9-9d54-4393-91f5-dd46f7758b4c service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Refreshing instance network info cache due to event network-changed-45a5a643-510a-45e2-a8e4-12eb53631daf. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ff31da45-c3aa-43fe-b52c-3c815dd7e7f4 req-21192af9-9d54-4393-91f5-dd46f7758b4c service nova] Acquiring lock "refresh_cache-970e81d2-c227-4eab-92dd-1ed732e67968" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Updating instance_info_cache with network_info: [{"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Releasing lock "refresh_cache-970e81d2-c227-4eab-92dd-1ed732e67968" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Instance network_info: |[{"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ff31da45-c3aa-43fe-b52c-3c815dd7e7f4 req-21192af9-9d54-4393-91f5-dd46f7758b4c service nova] Acquired lock "refresh_cache-970e81d2-c227-4eab-92dd-1ed732e67968" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:32:52 user nova-compute[71317]: DEBUG nova.network.neutron [req-ff31da45-c3aa-43fe-b52c-3c815dd7e7f4 req-21192af9-9d54-4393-91f5-dd46f7758b4c service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Refreshing network info cache for port 45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:32:53 user nova-compute[71317]: DEBUG nova.network.neutron [req-ff31da45-c3aa-43fe-b52c-3c815dd7e7f4 req-21192af9-9d54-4393-91f5-dd46f7758b4c service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Updated VIF entry in instance network info cache for port 45a5a643-510a-45e2-a8e4-12eb53631daf. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:32:53 user nova-compute[71317]: DEBUG nova.network.neutron [req-ff31da45-c3aa-43fe-b52c-3c815dd7e7f4 req-21192af9-9d54-4393-91f5-dd46f7758b4c service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Updating instance_info_cache with network_info: [{"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:32:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ff31da45-c3aa-43fe-b52c-3c815dd7e7f4 req-21192af9-9d54-4393-91f5-dd46f7758b4c service nova] Releasing lock "refresh_cache-970e81d2-c227-4eab-92dd-1ed732e67968" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:32:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG os_brick.utils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c25f785d-590c-4147-ab57-ecf708040cdc]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d5ac8bb-8033-47f4-94e1-ca7137f864dc]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6bcd390f-149d-4831-b2c2-8caa47e4ef59]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[21cbe62a-c2dc-4b8d-9da7-f28e26d11512]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:32:55 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:32:55 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b721830-01de-4d9c-9f52-e7e0563bf58a]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG os_brick.utils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] <== get_connector_properties: return (127ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:32:55 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Updating existing volume attachment record: 5ae6bc76-75a1-4f44-99a1-fdb9bf96b0ef {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:32:56 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446761.781248, ca085872-69ff-41e4-be13-ffaa9a7cca82 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:32:56 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] VM Stopped (Lifecycle Event)
Apr 14 04:32:57 user nova-compute[71317]: DEBUG nova.compute.manager [None req-6d590b9a-08ca-4231-91c8-43d713209f8a None None] [instance: ca085872-69ff-41e4-be13-ffaa9a7cca82] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:32:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:32:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:32:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:32:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:32:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:32:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:33:00 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Creating image(s)
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Ensure instance console log exists: /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Start _get_guest_xml network_info=[{"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'deafxyMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '970e81d2-c227-4eab-92dd-1ed732e67968', 'attached_at': '', 'detached_at': '', 'volume_id': '9108333c-dbbf-4edb-a2b5-c6f034073970', 'serial': '9108333c-dbbf-4edb-a2b5-c6f034073970'}, 'attachment_id': '5ae6bc76-75a1-4f44-99a1-fdb9bf96b0ef', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:33:00 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:33:00 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'deafxyMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:33:00 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:33:00 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:33:00 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7bdab2f-31ae-4907-8788-6e5f0170142b]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:33:00 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b62a7bb-0ff6-463d-bb1c-3b40624e7054]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970] added
Apr 14 04:33:00 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.102s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1de7b7c-5adf-4aa7-ae63-d6d99e4fc362]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[58a40846-4561-4452-a95b-2cfe70ae8f9e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v deafxyMo {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-56a3dea8-baaa-47e2-a7ea-1d8a2282276b req-24492837-67cc-4b98-943c-fd513ca452d6 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Received event network-changed-bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-56a3dea8-baaa-47e2-a7ea-1d8a2282276b req-24492837-67cc-4b98-943c-fd513ca452d6 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Refreshing instance network info cache due to event network-changed-bc83f476-5176-4301-8272-80034f4a0daf. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-56a3dea8-baaa-47e2-a7ea-1d8a2282276b req-24492837-67cc-4b98-943c-fd513ca452d6 service nova] Acquiring lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-56a3dea8-baaa-47e2-a7ea-1d8a2282276b req-24492837-67cc-4b98-943c-fd513ca452d6 service nova] Acquired lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG nova.network.neutron [req-56a3dea8-baaa-47e2-a7ea-1d8a2282276b req-24492837-67cc-4b98-943c-fd513ca452d6 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Refreshing network info cache for port bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v deafxyMo" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[82bb24d1-332c-49ca-9a6f-ab13a098034d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'deafxyMo'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ac9aea4-7bea-475a-a98f-36ac4937a2e3]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[43b56032-8973-4723-9dd2-4d3c535126eb]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:33:00 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:33:00 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:33:00 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:33:00 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --login" returned: 0 in 0.058s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1718884-0533-40b3-b305-0393fabfd082]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970, portal: 172.16.0.220,3260]
Apr 14 04:33:00 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970, portal: 172.16.0.220,3260] successful.
Apr 14 04:33:00 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[35810d43-2fe7-4728-bc74-6fe28accd33b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:00 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[45ad8739-d86b-4e73-8955-645e97411a9e]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:33:01 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:33:01 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:33:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:33:01 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:33:01 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:33:01 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.383s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] HCTL ('34', '-', '-', 0) found on session 44 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d21aa23e-b9ed-4f50-9d41-e03fd9ec9a20]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Searching for a device in session 44 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'deafxyMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG nova.network.neutron [req-56a3dea8-baaa-47e2-a7ea-1d8a2282276b req-24492837-67cc-4b98-943c-fd513ca452d6 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Updated VIF entry in instance network info cache for port bc83f476-5176-4301-8272-80034f4a0daf. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG nova.network.neutron [req-56a3dea8-baaa-47e2-a7ea-1d8a2282276b req-24492837-67cc-4b98-943c-fd513ca452d6 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Updating instance_info_cache with network_info: [{"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:33:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-56a3dea8-baaa-47e2-a7ea-1d8a2282276b req-24492837-67cc-4b98-943c-fd513ca452d6 service nova] Releasing lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.421s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] <== connect_volume: return (1423ms) {'type': 'block', 'scsi_wwn': '26334313830363633', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26334313830363633', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:32:48Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1167245313',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1167245313',id=35,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-djyaua3x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:32:50Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=970e81d2-c227-4eab-92dd-1ed732e67968,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:cd:43:7e,bridge_name='br-int',has_traffic_filtering=True,id=45a5a643-510a-45e2-a8e4-12eb53631daf,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap45a5a643-51') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG nova.objects.instance [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'pci_devices' on Instance uuid 970e81d2-c227-4eab-92dd-1ed732e67968 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:33:02 user nova-compute[71317]:   <uuid>970e81d2-c227-4eab-92dd-1ed732e67968</uuid>
Apr 14 04:33:02 user nova-compute[71317]:   <name>instance-00000023</name>
Apr 14 04:33:02 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:33:02 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:33:02 user nova-compute[71317]:   <metadata>
Apr 14 04:33:02 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:33:02 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:33:02 user nova-compute[71317]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1167245313</nova:name>
Apr 14 04:33:02 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:33:00</nova:creationTime>
Apr 14 04:33:02 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:33:02 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:33:02 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:33:02 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:33:02 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:33:02 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:33:02 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:33:02 user nova-compute[71317]:       <nova:owner>
Apr 14 04:33:02 user nova-compute[71317]:         <nova:user uuid="6ec7103e06e643a6b9b2c908ba8d7b06">tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member</nova:user>
Apr 14 04:33:02 user nova-compute[71317]:         <nova:project uuid="d5995b934b0546c0983387e46e3b3df4">tempest-ServerBootFromVolumeStableRescueTest-247821842</nova:project>
Apr 14 04:33:02 user nova-compute[71317]:       </nova:owner>
Apr 14 04:33:02 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:33:02 user nova-compute[71317]:       <nova:ports>
Apr 14 04:33:02 user nova-compute[71317]:         <nova:port uuid="45a5a643-510a-45e2-a8e4-12eb53631daf">
Apr 14 04:33:02 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 14 04:33:02 user nova-compute[71317]:         </nova:port>
Apr 14 04:33:02 user nova-compute[71317]:       </nova:ports>
Apr 14 04:33:02 user nova-compute[71317]:     </nova:instance>
Apr 14 04:33:02 user nova-compute[71317]:   </metadata>
Apr 14 04:33:02 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:33:02 user nova-compute[71317]:     <system>
Apr 14 04:33:02 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:33:02 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:33:02 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:33:02 user nova-compute[71317]:       <entry name="serial">970e81d2-c227-4eab-92dd-1ed732e67968</entry>
Apr 14 04:33:02 user nova-compute[71317]:       <entry name="uuid">970e81d2-c227-4eab-92dd-1ed732e67968</entry>
Apr 14 04:33:02 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:33:02 user nova-compute[71317]:     </system>
Apr 14 04:33:02 user nova-compute[71317]:   </sysinfo>
Apr 14 04:33:02 user nova-compute[71317]:   <os>
Apr 14 04:33:02 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:33:02 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:33:02 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:33:02 user nova-compute[71317]:   </os>
Apr 14 04:33:02 user nova-compute[71317]:   <features>
Apr 14 04:33:02 user nova-compute[71317]:     <acpi/>
Apr 14 04:33:02 user nova-compute[71317]:     <apic/>
Apr 14 04:33:02 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:33:02 user nova-compute[71317]:   </features>
Apr 14 04:33:02 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:33:02 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:33:02 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:33:02 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:33:02 user nova-compute[71317]:   </clock>
Apr 14 04:33:02 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:33:02 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:33:02 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:33:02 user nova-compute[71317]:   </cpu>
Apr 14 04:33:02 user nova-compute[71317]:   <devices>
Apr 14 04:33:02 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:33:02 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:33:02 user nova-compute[71317]:       <source dev="/dev/sdc"/>
Apr 14 04:33:02 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:33:02 user nova-compute[71317]:       <serial>9108333c-dbbf-4edb-a2b5-c6f034073970</serial>
Apr 14 04:33:02 user nova-compute[71317]:     </disk>
Apr 14 04:33:02 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:33:02 user nova-compute[71317]:       <mac address="fa:16:3e:cd:43:7e"/>
Apr 14 04:33:02 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:33:02 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:33:02 user nova-compute[71317]:       <target dev="tap45a5a643-51"/>
Apr 14 04:33:02 user nova-compute[71317]:     </interface>
Apr 14 04:33:02 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:33:02 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/console.log" append="off"/>
Apr 14 04:33:02 user nova-compute[71317]:     </serial>
Apr 14 04:33:02 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:33:02 user nova-compute[71317]:     <video>
Apr 14 04:33:02 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:33:02 user nova-compute[71317]:     </video>
Apr 14 04:33:02 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:33:02 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:33:02 user nova-compute[71317]:     </rng>
Apr 14 04:33:02 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:33:02 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:33:02 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:33:02 user nova-compute[71317]:     </memballoon>
Apr 14 04:33:02 user nova-compute[71317]:   </devices>
Apr 14 04:33:02 user nova-compute[71317]: </domain>
Apr 14 04:33:02 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:32:48Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1167245313',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1167245313',id=35,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-djyaua3x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:32:50Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=970e81d2-c227-4eab-92dd-1ed732e67968,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:cd:43:7e,bridge_name='br-int',has_traffic_filtering=True,id=45a5a643-510a-45e2-a8e4-12eb53631daf,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap45a5a643-51') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG os_vif [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:cd:43:7e,bridge_name='br-int',has_traffic_filtering=True,id=45a5a643-510a-45e2-a8e4-12eb53631daf,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap45a5a643-51') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap45a5a643-51, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap45a5a643-51, col_values=(('external_ids', {'iface-id': '45a5a643-510a-45e2-a8e4-12eb53631daf', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:cd:43:7e', 'vm-uuid': '970e81d2-c227-4eab-92dd-1ed732e67968'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:02 user nova-compute[71317]: INFO os_vif [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:cd:43:7e,bridge_name='br-int',has_traffic_filtering=True,id=45a5a643-510a-45e2-a8e4-12eb53631daf,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap45a5a643-51')
Apr 14 04:33:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:33:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No VIF found with MAC fa:16:3e:cd:43:7e, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:33:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:03 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446768.4687817, 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:33:03 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] VM Stopped (Lifecycle Event)
Apr 14 04:33:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5be8fdf4-d2a3-4d7c-a3d7-8be3b7629289 None None] [instance: 2f7e20cd-33ef-4f83-bcdb-a13b9b51370a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-a59271ec-f012-4533-847e-c4f2c622ca08 req-6018dd95-4580-4930-a28b-95b60d29a140 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a59271ec-f012-4533-847e-c4f2c622ca08 req-6018dd95-4580-4930-a28b-95b60d29a140 service nova] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a59271ec-f012-4533-847e-c4f2c622ca08 req-6018dd95-4580-4930-a28b-95b60d29a140 service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a59271ec-f012-4533-847e-c4f2c622ca08 req-6018dd95-4580-4930-a28b-95b60d29a140 service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-a59271ec-f012-4533-847e-c4f2c622ca08 req-6018dd95-4580-4930-a28b-95b60d29a140 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] No waiting events found dispatching network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:33:04 user nova-compute[71317]: WARNING nova.compute.manager [req-a59271ec-f012-4533-847e-c4f2c622ca08 req-6018dd95-4580-4930-a28b-95b60d29a140 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received unexpected event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf for instance with vm_state building and task_state spawning.
Apr 14 04:33:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:05 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446785.9940205, 970e81d2-c227-4eab-92dd-1ed732e67968 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:33:05 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] VM Resumed (Lifecycle Event)
Apr 14 04:33:05 user nova-compute[71317]: DEBUG nova.compute.manager [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:33:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:33:06 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Instance spawned successfully.
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:06 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446785.998313, 970e81d2-c227-4eab-92dd-1ed732e67968 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:33:06 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] VM Started (Lifecycle Event)
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:33:06 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:33:06 user nova-compute[71317]: INFO nova.compute.manager [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Took 5.57 seconds to spawn the instance on the hypervisor.
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:33:06 user nova-compute[71317]: INFO nova.compute.claims [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Claim successful on node user
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-5d8d1670-0aea-49e8-a44d-553e6df06a97 req-7db0ff28-8631-4ff7-988c-973f37764cea service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5d8d1670-0aea-49e8-a44d-553e6df06a97 req-7db0ff28-8631-4ff7-988c-973f37764cea service nova] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5d8d1670-0aea-49e8-a44d-553e6df06a97 req-7db0ff28-8631-4ff7-988c-973f37764cea service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5d8d1670-0aea-49e8-a44d-553e6df06a97 req-7db0ff28-8631-4ff7-988c-973f37764cea service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-5d8d1670-0aea-49e8-a44d-553e6df06a97 req-7db0ff28-8631-4ff7-988c-973f37764cea service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] No waiting events found dispatching network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:33:06 user nova-compute[71317]: WARNING nova.compute.manager [req-5d8d1670-0aea-49e8-a44d-553e6df06a97 req-7db0ff28-8631-4ff7-988c-973f37764cea service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received unexpected event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf for instance with vm_state building and task_state spawning.
Apr 14 04:33:06 user nova-compute[71317]: INFO nova.compute.manager [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Took 17.02 seconds to build instance.
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-698c32a3-4a57-4bc4-9e50-b4aea652d7bf tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "970e81d2-c227-4eab-92dd-1ed732e67968" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 17.129s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.395s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.network.neutron [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:33:06 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.compute.manager [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:33:06 user nova-compute[71317]: INFO nova.virt.block_device [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Booting with volume 992a3f86-d313-4b1e-9491-3008be037269 at /dev/vda
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.policy [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e35b3ae371cf4dd4b21e2999e58cd2f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c14c332dfcc44b49ddf3cb482dd0d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG os_brick.utils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2d8a468-9879-42c6-b800-ab92e632798f]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8edb180-8e4f-484d-9ae8-d7cd8c9e0700]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d90429c7-0533-4d9e-9749-ab8717be6742]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5296a7c-e029-44b5-afca-0079b206ca4b]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:33:06 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:33:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d5981b6-9109-43ed-8163-64e7634bfc53]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG os_brick.utils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== get_connector_properties: return (133ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:33:06 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updating existing volume attachment record: bd68b9f6-ee70-4c4e-b98b-a28568ab823f {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:33:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:07 user nova-compute[71317]: DEBUG nova.network.neutron [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Successfully created port: 3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG nova.network.neutron [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Successfully updated port: 3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquired lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG nova.network.neutron [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-43d799e7-f6bf-45f4-9df1-ef756b4c0fa3 req-22de7e60-8c79-41cd-b316-7542e21985f7 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received event network-changed-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-43d799e7-f6bf-45f4-9df1-ef756b4c0fa3 req-22de7e60-8c79-41cd-b316-7542e21985f7 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Refreshing instance network info cache due to event network-changed-3562cd0d-8f80-4113-a397-b56e4455ef4a. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-43d799e7-f6bf-45f4-9df1-ef756b4c0fa3 req-22de7e60-8c79-41cd-b316-7542e21985f7 service nova] Acquiring lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG nova.network.neutron [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG nova.network.neutron [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updating instance_info_cache with network_info: [{"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Releasing lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Instance network_info: |[{"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-43d799e7-f6bf-45f4-9df1-ef756b4c0fa3 req-22de7e60-8c79-41cd-b316-7542e21985f7 service nova] Acquired lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:33:08 user nova-compute[71317]: DEBUG nova.network.neutron [req-43d799e7-f6bf-45f4-9df1-ef756b4c0fa3 req-22de7e60-8c79-41cd-b316-7542e21985f7 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Refreshing network info cache for port 3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:33:09 user nova-compute[71317]: DEBUG nova.network.neutron [req-43d799e7-f6bf-45f4-9df1-ef756b4c0fa3 req-22de7e60-8c79-41cd-b316-7542e21985f7 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updated VIF entry in instance network info cache for port 3562cd0d-8f80-4113-a397-b56e4455ef4a. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:33:09 user nova-compute[71317]: DEBUG nova.network.neutron [req-43d799e7-f6bf-45f4-9df1-ef756b4c0fa3 req-22de7e60-8c79-41cd-b316-7542e21985f7 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updating instance_info_cache with network_info: [{"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:33:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-43d799e7-f6bf-45f4-9df1-ef756b4c0fa3 req-22de7e60-8c79-41cd-b316-7542e21985f7 service nova] Releasing lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.compute.manager [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:33:11 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Creating image(s)
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Ensure instance console log exists: /opt/stack/data/nova/instances/695d931b-1f30-4d57-b6d7-d199d2874e65/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Start _get_guest_xml network_info=[{"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': True, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sPgLhMjz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '695d931b-1f30-4d57-b6d7-d199d2874e65', 'attached_at': '', 'detached_at': '', 'volume_id': '992a3f86-d313-4b1e-9491-3008be037269', 'serial': '992a3f86-d313-4b1e-9491-3008be037269'}, 'attachment_id': 'bd68b9f6-ee70-4c4e-b98b-a28568ab823f', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:33:11 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:33:11 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sPgLhMjz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:33:11 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:33:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:33:11 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:33:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:33:11 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[186f96dc-6c9b-4ad7-a958-d56cad751e23]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:33:11 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[35cdc038-a980-4fd5-b267-4c0ce03518bd]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269] added
Apr 14 04:33:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d14b203d-98d1-4e98-b0ec-cf7b8a78ea95]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb663193-5f23-4304-9b95-4bb9a88fdcda]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v sPgLhMjz {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v sPgLhMjz" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f13ba5da-212c-4f33-ba1e-3e930f78edf4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'sPgLhMjz'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.042s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f347d5a1-2724-47cb-9d86-a4aaddaa55f5]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb1efbac-a139-4690-b925-dedac472d36c]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --login" returned: 0 in 0.074s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[945e504e-e9a1-460d-8f46-399010cbb48c]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260]
Apr 14 04:33:12 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260] successful.
Apr 14 04:33:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6ac9ba52-712e-4efa-acae-affe46afc275]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1cc881d4-acb5-49a8-8589-221f25c0e032]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:33:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.327s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] HCTL ('35', '-', '-', 0) found on session 45 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a3efa4c-2873-4f23-bb57-13cfadad21cf]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Searching for a device in session 45 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sPgLhMjz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:33:12 user nova-compute[71317]: INFO nova.compute.manager [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] instance snapshotting
Apr 14 04:33:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Beginning live snapshot process
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json -f qcow2 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json -f qcow2" returned: 0 in 0.158s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json -f qcow2 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json -f qcow2" returned: 0 in 0.167s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp9k459uwo/22118fea1fd640e38f2b754dff153f02.delta 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp9k459uwo/22118fea1fd640e38f2b754dff153f02.delta 1073741824" returned: 0 in 0.052s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Quiescing instance not available: QEMU guest agent is not enabled.
Apr 14 04:33:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.368s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== connect_volume: return (1369ms) {'type': 'block', 'scsi_wwn': '26531633364303339', 'path': '/dev/sdd'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26531633364303339', 'path': '/dev/sdd'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:33:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-372327292',display_name='tempest-ServerActionsTestJSON-server-372327292',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-372327292',id=36,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-uxrcwktp',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:33:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=695d931b-1f30-4d57-b6d7-d199d2874e65,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG nova.objects.instance [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'pci_devices' on Instance uuid 695d931b-1f30-4d57-b6d7-d199d2874e65 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:33:13 user nova-compute[71317]:   <uuid>695d931b-1f30-4d57-b6d7-d199d2874e65</uuid>
Apr 14 04:33:13 user nova-compute[71317]:   <name>instance-00000024</name>
Apr 14 04:33:13 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:33:13 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:33:13 user nova-compute[71317]:   <metadata>
Apr 14 04:33:13 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:33:13 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:33:13 user nova-compute[71317]:       <nova:name>tempest-ServerActionsTestJSON-server-372327292</nova:name>
Apr 14 04:33:13 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:33:11</nova:creationTime>
Apr 14 04:33:13 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:33:13 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:33:13 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:33:13 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:33:13 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:33:13 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:33:13 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:33:13 user nova-compute[71317]:       <nova:owner>
Apr 14 04:33:13 user nova-compute[71317]:         <nova:user uuid="e35b3ae371cf4dd4b21e2999e58cd2f4">tempest-ServerActionsTestJSON-1048786641-project-member</nova:user>
Apr 14 04:33:13 user nova-compute[71317]:         <nova:project uuid="8c14c332dfcc44b49ddf3cb482dd0d4c">tempest-ServerActionsTestJSON-1048786641</nova:project>
Apr 14 04:33:13 user nova-compute[71317]:       </nova:owner>
Apr 14 04:33:13 user nova-compute[71317]:       <nova:ports>
Apr 14 04:33:13 user nova-compute[71317]:         <nova:port uuid="3562cd0d-8f80-4113-a397-b56e4455ef4a">
Apr 14 04:33:13 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 14 04:33:13 user nova-compute[71317]:         </nova:port>
Apr 14 04:33:13 user nova-compute[71317]:       </nova:ports>
Apr 14 04:33:13 user nova-compute[71317]:     </nova:instance>
Apr 14 04:33:13 user nova-compute[71317]:   </metadata>
Apr 14 04:33:13 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:33:13 user nova-compute[71317]:     <system>
Apr 14 04:33:13 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:33:13 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:33:13 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:33:13 user nova-compute[71317]:       <entry name="serial">695d931b-1f30-4d57-b6d7-d199d2874e65</entry>
Apr 14 04:33:13 user nova-compute[71317]:       <entry name="uuid">695d931b-1f30-4d57-b6d7-d199d2874e65</entry>
Apr 14 04:33:13 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:33:13 user nova-compute[71317]:     </system>
Apr 14 04:33:13 user nova-compute[71317]:   </sysinfo>
Apr 14 04:33:13 user nova-compute[71317]:   <os>
Apr 14 04:33:13 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:33:13 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:33:13 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:33:13 user nova-compute[71317]:   </os>
Apr 14 04:33:13 user nova-compute[71317]:   <features>
Apr 14 04:33:13 user nova-compute[71317]:     <acpi/>
Apr 14 04:33:13 user nova-compute[71317]:     <apic/>
Apr 14 04:33:13 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:33:13 user nova-compute[71317]:   </features>
Apr 14 04:33:13 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:33:13 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:33:13 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:33:13 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:33:13 user nova-compute[71317]:   </clock>
Apr 14 04:33:13 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:33:13 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:33:13 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:33:13 user nova-compute[71317]:   </cpu>
Apr 14 04:33:13 user nova-compute[71317]:   <devices>
Apr 14 04:33:13 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:33:13 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:33:13 user nova-compute[71317]:       <source dev="/dev/sdd"/>
Apr 14 04:33:13 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:33:13 user nova-compute[71317]:       <serial>992a3f86-d313-4b1e-9491-3008be037269</serial>
Apr 14 04:33:13 user nova-compute[71317]:     </disk>
Apr 14 04:33:13 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:33:13 user nova-compute[71317]:       <mac address="fa:16:3e:f7:65:55"/>
Apr 14 04:33:13 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:33:13 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:33:13 user nova-compute[71317]:       <target dev="tap3562cd0d-8f"/>
Apr 14 04:33:13 user nova-compute[71317]:     </interface>
Apr 14 04:33:13 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:33:13 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/695d931b-1f30-4d57-b6d7-d199d2874e65/console.log" append="off"/>
Apr 14 04:33:13 user nova-compute[71317]:     </serial>
Apr 14 04:33:13 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:33:13 user nova-compute[71317]:     <video>
Apr 14 04:33:13 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:33:13 user nova-compute[71317]:     </video>
Apr 14 04:33:13 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:33:13 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:33:13 user nova-compute[71317]:     </rng>
Apr 14 04:33:13 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:33:13 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:33:13 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:33:13 user nova-compute[71317]:     </memballoon>
Apr 14 04:33:13 user nova-compute[71317]:   </devices>
Apr 14 04:33:13 user nova-compute[71317]: </domain>
Apr 14 04:33:13 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:33:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-372327292',display_name='tempest-ServerActionsTestJSON-server-372327292',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-372327292',id=36,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-uxrcwktp',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:33:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=695d931b-1f30-4d57-b6d7-d199d2874e65,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG os_vif [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3562cd0d-8f, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap3562cd0d-8f, col_values=(('external_ids', {'iface-id': '3562cd0d-8f80-4113-a397-b56e4455ef4a', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f7:65:55', 'vm-uuid': '695d931b-1f30-4d57-b6d7-d199d2874e65'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:13 user nova-compute[71317]: INFO os_vif [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f')
Apr 14 04:33:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No VIF found with MAC fa:16:3e:f7:65:55, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:33:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] COPY block job progress, current cursor: 0 final cursor: 43843584 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:33:14 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] COPY block job progress, current cursor: 43843584 final cursor: 43843584 {{(pid=71317) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 04:33:14 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Skipping quiescing instance: QEMU guest agent is not enabled.
Apr 14 04:33:14 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:33:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp9k459uwo/22118fea1fd640e38f2b754dff153f02.delta /opt/stack/data/nova/instances/snapshots/tmp9k459uwo/22118fea1fd640e38f2b754dff153f02 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp9k459uwo/22118fea1fd640e38f2b754dff153f02.delta /opt/stack/data/nova/instances/snapshots/tmp9k459uwo/22118fea1fd640e38f2b754dff153f02" returned: 0 in 0.443s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:15 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Snapshot extracted, beginning image upload
Apr 14 04:33:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f3513d0-ad11-48fa-805e-f33c983a8461 req-017b7883-854c-4a39-b04a-7a14582e9f14 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received event network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f3513d0-ad11-48fa-805e-f33c983a8461 req-017b7883-854c-4a39-b04a-7a14582e9f14 service nova] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f3513d0-ad11-48fa-805e-f33c983a8461 req-017b7883-854c-4a39-b04a-7a14582e9f14 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f3513d0-ad11-48fa-805e-f33c983a8461 req-017b7883-854c-4a39-b04a-7a14582e9f14 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:15 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f3513d0-ad11-48fa-805e-f33c983a8461 req-017b7883-854c-4a39-b04a-7a14582e9f14 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] No waiting events found dispatching network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:33:15 user nova-compute[71317]: WARNING nova.compute.manager [req-5f3513d0-ad11-48fa-805e-f33c983a8461 req-017b7883-854c-4a39-b04a-7a14582e9f14 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received unexpected event network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a for instance with vm_state building and task_state spawning.
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446797.1199083, 695d931b-1f30-4d57-b6d7-d199d2874e65 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:33:17 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] VM Resumed (Lifecycle Event)
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:33:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Instance spawned successfully.
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:33:17 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446797.1213222, 695d931b-1f30-4d57-b6d7-d199d2874e65 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:33:17 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] VM Started (Lifecycle Event)
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:33:17 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-9581b2e2-ec8a-48cf-a179-96b87a1d338e req-1ae82a6c-c867-4185-861e-bed03abf3dc3 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received event network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9581b2e2-ec8a-48cf-a179-96b87a1d338e req-1ae82a6c-c867-4185-861e-bed03abf3dc3 service nova] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9581b2e2-ec8a-48cf-a179-96b87a1d338e req-1ae82a6c-c867-4185-861e-bed03abf3dc3 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-9581b2e2-ec8a-48cf-a179-96b87a1d338e req-1ae82a6c-c867-4185-861e-bed03abf3dc3 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-9581b2e2-ec8a-48cf-a179-96b87a1d338e req-1ae82a6c-c867-4185-861e-bed03abf3dc3 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] No waiting events found dispatching network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:33:17 user nova-compute[71317]: WARNING nova.compute.manager [req-9581b2e2-ec8a-48cf-a179-96b87a1d338e req-1ae82a6c-c867-4185-861e-bed03abf3dc3 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received unexpected event network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a for instance with vm_state building and task_state spawning.
Apr 14 04:33:17 user nova-compute[71317]: INFO nova.compute.manager [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Took 5.52 seconds to spawn the instance on the hypervisor.
Apr 14 04:33:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Snapshot image upload complete
Apr 14 04:33:17 user nova-compute[71317]: INFO nova.compute.manager [None req-6e135e9f-4154-4a19-8a68-db37eb18299d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Took 5.12 seconds to snapshot the instance on the hypervisor.
Apr 14 04:33:17 user nova-compute[71317]: INFO nova.compute.manager [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Took 11.27 seconds to build instance.
Apr 14 04:33:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-62db2ca5-1eb7-4991-810b-64f604e00e3e tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 11.379s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:33:23 user nova-compute[71317]: INFO nova.compute.claims [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Claim successful on node user
Apr 14 04:33:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.559s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:33:23 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:33:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG nova.policy [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ebe3f41b95824cd78193d0bbe206a66b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e6175a4372245a4a57dca1f12f7f007', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:33:23 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Creating image(s)
Apr 14 04:33:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "/opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "/opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "/opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "aafe06d5ca392c133765c9fd1ef5f8d77b20d6af" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "aafe06d5ca392c133765c9fd1ef5f8d77b20d6af" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af.part --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:24 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Successfully created port: 52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:33:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af.part --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:24 user nova-compute[71317]: DEBUG nova.virt.images [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] 8666f9e6-ed56-46cf-a3bc-a12038eea805 was qcow2, converting to raw {{(pid=71317) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 04:33:24 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:33:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af.part /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af.converted {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af.part /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af.converted" returned: 0 in 0.398s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af.converted --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af.converted --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "aafe06d5ca392c133765c9fd1ef5f8d77b20d6af" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.436s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Successfully updated port: 52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "aafe06d5ca392c133765c9fd1ef5f8d77b20d6af" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "aafe06d5ca392c133765c9fd1ef5f8d77b20d6af" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "refresh_cache-3746b520-c10a-413e-9f1f-8807b3fd7c9a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquired lock "refresh_cache-3746b520-c10a-413e-9f1f-8807b3fd7c9a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-c6a15206-2d32-4969-8942-16be03f22c6a req-aeaa3f0a-a48e-4296-97f2-a3cd4383da3e service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Received event network-changed-52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-c6a15206-2d32-4969-8942-16be03f22c6a req-aeaa3f0a-a48e-4296-97f2-a3cd4383da3e service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Refreshing instance network info cache due to event network-changed-52608de5-fde8-4fd9-81c0-10a060ab7130. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c6a15206-2d32-4969-8942-16be03f22c6a req-aeaa3f0a-a48e-4296-97f2-a3cd4383da3e service nova] Acquiring lock "refresh_cache-3746b520-c10a-413e-9f1f-8807b3fd7c9a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af,backing_fmt=raw /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af,backing_fmt=raw /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk 1073741824" returned: 0 in 0.048s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "aafe06d5ca392c133765c9fd1ef5f8d77b20d6af" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.193s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af --force-share --output=json" returned: 0 in 0.130s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.objects.instance [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lazy-loading 'migration_context' on Instance uuid 3746b520-c10a-413e-9f1f-8807b3fd7c9a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Ensure instance console log exists: /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Updating instance_info_cache with network_info: [{"id": "52608de5-fde8-4fd9-81c0-10a060ab7130", "address": "fa:16:3e:36:5c:89", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52608de5-fd", "ovs_interfaceid": "52608de5-fde8-4fd9-81c0-10a060ab7130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Releasing lock "refresh_cache-3746b520-c10a-413e-9f1f-8807b3fd7c9a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Instance network_info: |[{"id": "52608de5-fde8-4fd9-81c0-10a060ab7130", "address": "fa:16:3e:36:5c:89", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52608de5-fd", "ovs_interfaceid": "52608de5-fde8-4fd9-81c0-10a060ab7130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c6a15206-2d32-4969-8942-16be03f22c6a req-aeaa3f0a-a48e-4296-97f2-a3cd4383da3e service nova] Acquired lock "refresh_cache-3746b520-c10a-413e-9f1f-8807b3fd7c9a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.network.neutron [req-c6a15206-2d32-4969-8942-16be03f22c6a req-aeaa3f0a-a48e-4296-97f2-a3cd4383da3e service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Refreshing network info cache for port 52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Start _get_guest_xml network_info=[{"id": "52608de5-fde8-4fd9-81c0-10a060ab7130", "address": "fa:16:3e:36:5c:89", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52608de5-fd", "ovs_interfaceid": "52608de5-fde8-4fd9-81c0-10a060ab7130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='63b448e7305019c82bf4ca4eb9503984',container_format='bare',created_at=2023-04-14T04:33:12Z,direct_url=<?>,disk_format='qcow2',id=8666f9e6-ed56-46cf-a3bc-a12038eea805,min_disk=1,min_ram=0,name='tempest-TestStampPatternsnapshot-153401794',owner='0e6175a4372245a4a57dca1f12f7f007',properties=ImageMetaProps,protected=<?>,size=38666240,status='active',tags=<?>,updated_at=2023-04-14T04:33:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '8666f9e6-ed56-46cf-a3bc-a12038eea805'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:33:25 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:33:25 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='63b448e7305019c82bf4ca4eb9503984',container_format='bare',created_at=2023-04-14T04:33:12Z,direct_url=<?>,disk_format='qcow2',id=8666f9e6-ed56-46cf-a3bc-a12038eea805,min_disk=1,min_ram=0,name='tempest-TestStampPatternsnapshot-153401794',owner='0e6175a4372245a4a57dca1f12f7f007',properties=ImageMetaProps,protected=<?>,size=38666240,status='active',tags=<?>,updated_at=2023-04-14T04:33:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:33:22Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-1660505126',display_name='tempest-TestStampPattern-server-1660505126',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-1660505126',id=37,image_ref='8666f9e6-ed56-46cf-a3bc-a12038eea805',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPy10nG4Vr3ssb6R59u94cqlTShzxPhKC2bNuw8xGLQYLxJf5JqesyHfE6AT1NUBBqZ5yv5gwPvV4khi3Sx9VWvIr6XsmH3ZmGby83ZqN/gypVHL9c9HqBsp1SNADUtbzA==',key_name='tempest-TestStampPattern-1877604753',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0e6175a4372245a4a57dca1f12f7f007',ramdisk_id='',reservation_id='r-w2epmlt6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_boot_roles='member,reader',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='390ca845-5660-49d0-bd8a-fcfc39176ca5',image_min_disk='1',image_min_ram='0',image_owner_id='0e6175a4372245a4a57dca1f12f7f007',image_owner_project_name='tempest-TestStampPattern-267011728',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-267011728-project-member',image_user_id='ebe3f41b95824cd78193d0bbe206a66b',network_allocated='True',owner_project_name='tempest-TestStampPattern-267011728',owner_user_name='tempest-TestStampPattern-267011728-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:33:24Z,user_data=None,user_id='ebe3f41b95824cd78193d0bbe206a66b',uuid=3746b520-c10a-413e-9f1f-8807b3fd7c9a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "52608de5-fde8-4fd9-81c0-10a060ab7130", "address": "fa:16:3e:36:5c:89", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52608de5-fd", "ovs_interfaceid": "52608de5-fde8-4fd9-81c0-10a060ab7130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Converting VIF {"id": "52608de5-fde8-4fd9-81c0-10a060ab7130", "address": "fa:16:3e:36:5c:89", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52608de5-fd", "ovs_interfaceid": "52608de5-fde8-4fd9-81c0-10a060ab7130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:36:5c:89,bridge_name='br-int',has_traffic_filtering=True,id=52608de5-fde8-4fd9-81c0-10a060ab7130,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52608de5-fd') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.objects.instance [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lazy-loading 'pci_devices' on Instance uuid 3746b520-c10a-413e-9f1f-8807b3fd7c9a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:33:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:33:25 user nova-compute[71317]:   <uuid>3746b520-c10a-413e-9f1f-8807b3fd7c9a</uuid>
Apr 14 04:33:25 user nova-compute[71317]:   <name>instance-00000025</name>
Apr 14 04:33:25 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:33:25 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:33:25 user nova-compute[71317]:   <metadata>
Apr 14 04:33:25 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:33:25 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:33:25 user nova-compute[71317]:       <nova:name>tempest-TestStampPattern-server-1660505126</nova:name>
Apr 14 04:33:25 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:33:25</nova:creationTime>
Apr 14 04:33:25 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:33:25 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:33:25 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:33:25 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:33:25 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:33:25 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:33:25 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:33:25 user nova-compute[71317]:       <nova:owner>
Apr 14 04:33:25 user nova-compute[71317]:         <nova:user uuid="ebe3f41b95824cd78193d0bbe206a66b">tempest-TestStampPattern-267011728-project-member</nova:user>
Apr 14 04:33:25 user nova-compute[71317]:         <nova:project uuid="0e6175a4372245a4a57dca1f12f7f007">tempest-TestStampPattern-267011728</nova:project>
Apr 14 04:33:25 user nova-compute[71317]:       </nova:owner>
Apr 14 04:33:25 user nova-compute[71317]:       <nova:root type="image" uuid="8666f9e6-ed56-46cf-a3bc-a12038eea805"/>
Apr 14 04:33:25 user nova-compute[71317]:       <nova:ports>
Apr 14 04:33:25 user nova-compute[71317]:         <nova:port uuid="52608de5-fde8-4fd9-81c0-10a060ab7130">
Apr 14 04:33:25 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 04:33:25 user nova-compute[71317]:         </nova:port>
Apr 14 04:33:25 user nova-compute[71317]:       </nova:ports>
Apr 14 04:33:25 user nova-compute[71317]:     </nova:instance>
Apr 14 04:33:25 user nova-compute[71317]:   </metadata>
Apr 14 04:33:25 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:33:25 user nova-compute[71317]:     <system>
Apr 14 04:33:25 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:33:25 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:33:25 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:33:25 user nova-compute[71317]:       <entry name="serial">3746b520-c10a-413e-9f1f-8807b3fd7c9a</entry>
Apr 14 04:33:25 user nova-compute[71317]:       <entry name="uuid">3746b520-c10a-413e-9f1f-8807b3fd7c9a</entry>
Apr 14 04:33:25 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:33:25 user nova-compute[71317]:     </system>
Apr 14 04:33:25 user nova-compute[71317]:   </sysinfo>
Apr 14 04:33:25 user nova-compute[71317]:   <os>
Apr 14 04:33:25 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:33:25 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:33:25 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:33:25 user nova-compute[71317]:   </os>
Apr 14 04:33:25 user nova-compute[71317]:   <features>
Apr 14 04:33:25 user nova-compute[71317]:     <acpi/>
Apr 14 04:33:25 user nova-compute[71317]:     <apic/>
Apr 14 04:33:25 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:33:25 user nova-compute[71317]:   </features>
Apr 14 04:33:25 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:33:25 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:33:25 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:33:25 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:33:25 user nova-compute[71317]:   </clock>
Apr 14 04:33:25 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:33:25 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:33:25 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:33:25 user nova-compute[71317]:   </cpu>
Apr 14 04:33:25 user nova-compute[71317]:   <devices>
Apr 14 04:33:25 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:33:25 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:33:25 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk"/>
Apr 14 04:33:25 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:33:25 user nova-compute[71317]:     </disk>
Apr 14 04:33:25 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:33:25 user nova-compute[71317]:       <mac address="fa:16:3e:36:5c:89"/>
Apr 14 04:33:25 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:33:25 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:33:25 user nova-compute[71317]:       <target dev="tap52608de5-fd"/>
Apr 14 04:33:25 user nova-compute[71317]:     </interface>
Apr 14 04:33:25 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:33:25 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/console.log" append="off"/>
Apr 14 04:33:25 user nova-compute[71317]:     </serial>
Apr 14 04:33:25 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:33:25 user nova-compute[71317]:     <video>
Apr 14 04:33:25 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:33:25 user nova-compute[71317]:     </video>
Apr 14 04:33:25 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:33:25 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:33:25 user nova-compute[71317]:     </rng>
Apr 14 04:33:25 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:33:25 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:33:25 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:33:25 user nova-compute[71317]:     </memballoon>
Apr 14 04:33:25 user nova-compute[71317]:   </devices>
Apr 14 04:33:25 user nova-compute[71317]: </domain>
Apr 14 04:33:25 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:33:22Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-1660505126',display_name='tempest-TestStampPattern-server-1660505126',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-1660505126',id=37,image_ref='8666f9e6-ed56-46cf-a3bc-a12038eea805',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPy10nG4Vr3ssb6R59u94cqlTShzxPhKC2bNuw8xGLQYLxJf5JqesyHfE6AT1NUBBqZ5yv5gwPvV4khi3Sx9VWvIr6XsmH3ZmGby83ZqN/gypVHL9c9HqBsp1SNADUtbzA==',key_name='tempest-TestStampPattern-1877604753',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0e6175a4372245a4a57dca1f12f7f007',ramdisk_id='',reservation_id='r-w2epmlt6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_boot_roles='member,reader',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='390ca845-5660-49d0-bd8a-fcfc39176ca5',image_min_disk='1',image_min_ram='0',image_owner_id='0e6175a4372245a4a57dca1f12f7f007',image_owner_project_name='tempest-TestStampPattern-267011728',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-267011728-project-member',image_user_id='ebe3f41b95824cd78193d0bbe206a66b',network_allocated='True',owner_project_name='tempest-TestStampPattern-267011728',owner_user_name='tempest-TestStampPattern-267011728-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:33:24Z,user_data=None,user_id='ebe3f41b95824cd78193d0bbe206a66b',uuid=3746b520-c10a-413e-9f1f-8807b3fd7c9a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "52608de5-fde8-4fd9-81c0-10a060ab7130", "address": "fa:16:3e:36:5c:89", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52608de5-fd", "ovs_interfaceid": "52608de5-fde8-4fd9-81c0-10a060ab7130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Converting VIF {"id": "52608de5-fde8-4fd9-81c0-10a060ab7130", "address": "fa:16:3e:36:5c:89", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52608de5-fd", "ovs_interfaceid": "52608de5-fde8-4fd9-81c0-10a060ab7130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:36:5c:89,bridge_name='br-int',has_traffic_filtering=True,id=52608de5-fde8-4fd9-81c0-10a060ab7130,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52608de5-fd') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG os_vif [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:36:5c:89,bridge_name='br-int',has_traffic_filtering=True,id=52608de5-fde8-4fd9-81c0-10a060ab7130,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52608de5-fd') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap52608de5-fd, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap52608de5-fd, col_values=(('external_ids', {'iface-id': '52608de5-fde8-4fd9-81c0-10a060ab7130', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:36:5c:89', 'vm-uuid': '3746b520-c10a-413e-9f1f-8807b3fd7c9a'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:26 user nova-compute[71317]: INFO os_vif [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:36:5c:89,bridge_name='br-int',has_traffic_filtering=True,id=52608de5-fde8-4fd9-81c0-10a060ab7130,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52608de5-fd')
Apr 14 04:33:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] No VIF found with MAC fa:16:3e:36:5c:89, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG nova.network.neutron [req-c6a15206-2d32-4969-8942-16be03f22c6a req-aeaa3f0a-a48e-4296-97f2-a3cd4383da3e service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Updated VIF entry in instance network info cache for port 52608de5-fde8-4fd9-81c0-10a060ab7130. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG nova.network.neutron [req-c6a15206-2d32-4969-8942-16be03f22c6a req-aeaa3f0a-a48e-4296-97f2-a3cd4383da3e service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Updating instance_info_cache with network_info: [{"id": "52608de5-fde8-4fd9-81c0-10a060ab7130", "address": "fa:16:3e:36:5c:89", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52608de5-fd", "ovs_interfaceid": "52608de5-fde8-4fd9-81c0-10a060ab7130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c6a15206-2d32-4969-8942-16be03f22c6a req-aeaa3f0a-a48e-4296-97f2-a3cd4383da3e service nova] Releasing lock "refresh_cache-3746b520-c10a-413e-9f1f-8807b3fd7c9a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:33:26 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_power_states {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 3df97db3-fd6c-4be6-af32-7cde9dbcce75 {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid af7befec-3f77-4eb7-be53-d1b1e791713e {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 19a166de-f0c9-43f2-8c88-585e0d5d22cc {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 390ca845-5660-49d0-bd8a-fcfc39176ca5 {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 970e81d2-c227-4eab-92dd-1ed732e67968 {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 695d931b-1f30-4d57-b6d7-d199d2874e65 {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 3746b520-c10a-413e-9f1f-8807b3fd7c9a {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "af7befec-3f77-4eb7-be53-d1b1e791713e" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "970e81d2-c227-4eab-92dd-1ed732e67968" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.038s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.068s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.067s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.068s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.074s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.091s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.096s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "970e81d2-c227-4eab-92dd-1ed732e67968" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.099s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.101s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-30573da4-2afa-4a61-aaa9-d6429fa3fb86 req-ba0c295d-d764-4ddc-a537-9be4fd9237cb service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Received event network-vif-plugged-52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-30573da4-2afa-4a61-aaa9-d6429fa3fb86 req-ba0c295d-d764-4ddc-a537-9be4fd9237cb service nova] Acquiring lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-30573da4-2afa-4a61-aaa9-d6429fa3fb86 req-ba0c295d-d764-4ddc-a537-9be4fd9237cb service nova] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-30573da4-2afa-4a61-aaa9-d6429fa3fb86 req-ba0c295d-d764-4ddc-a537-9be4fd9237cb service nova] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-30573da4-2afa-4a61-aaa9-d6429fa3fb86 req-ba0c295d-d764-4ddc-a537-9be4fd9237cb service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] No waiting events found dispatching network-vif-plugged-52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:33:27 user nova-compute[71317]: WARNING nova.compute.manager [req-30573da4-2afa-4a61-aaa9-d6429fa3fb86 req-ba0c295d-d764-4ddc-a537-9be4fd9237cb service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Received unexpected event network-vif-plugged-52608de5-fde8-4fd9-81c0-10a060ab7130 for instance with vm_state building and task_state spawning.
Apr 14 04:33:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446809.606202, 3746b520-c10a-413e-9f1f-8807b3fd7c9a => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:33:29 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] VM Resumed (Lifecycle Event)
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:33:29 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Instance spawned successfully.
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:33:29 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446809.607423, 3746b520-c10a-413e-9f1f-8807b3fd7c9a => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:33:29 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] VM Started (Lifecycle Event)
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:33:29 user nova-compute[71317]: INFO nova.compute.manager [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Took 5.82 seconds to spawn the instance on the hypervisor.
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:29 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:33:29 user nova-compute[71317]: INFO nova.compute.manager [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Took 6.70 seconds to build instance.
Apr 14 04:33:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e2e9ae1-d3d8-4116-8256-281e04defca5 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.793s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 2.005s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:29 user nova-compute[71317]: INFO nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:33:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.compute.manager [req-0aa5abd8-aa21-4163-b394-b458b374a26e req-5605d71a-befe-404f-ab3f-b0b841465f2b service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Received event network-vif-plugged-52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0aa5abd8-aa21-4163-b394-b458b374a26e req-5605d71a-befe-404f-ab3f-b0b841465f2b service nova] Acquiring lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0aa5abd8-aa21-4163-b394-b458b374a26e req-5605d71a-befe-404f-ab3f-b0b841465f2b service nova] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0aa5abd8-aa21-4163-b394-b458b374a26e req-5605d71a-befe-404f-ab3f-b0b841465f2b service nova] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:29 user nova-compute[71317]: DEBUG nova.compute.manager [req-0aa5abd8-aa21-4163-b394-b458b374a26e req-5605d71a-befe-404f-ab3f-b0b841465f2b service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] No waiting events found dispatching network-vif-plugged-52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:33:29 user nova-compute[71317]: WARNING nova.compute.manager [req-0aa5abd8-aa21-4163-b394-b458b374a26e req-5605d71a-befe-404f-ab3f-b0b841465f2b service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Received unexpected event network-vif-plugged-52608de5-fde8-4fd9-81c0-10a060ab7130 for instance with vm_state active and task_state None.
Apr 14 04:33:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:33:33 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:33:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:34 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:33:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:33:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:33:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:33:34 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:33:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "af7befec-3f77-4eb7-be53-d1b1e791713e" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:34 user nova-compute[71317]: INFO nova.compute.manager [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Terminating instance
Apr 14 04:33:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-28bb7ed2-9ead-4393-9f65-b8638546b0f8 req-238a23f8-e1a8-4d08-baa5-0cc430e7f8a6 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Received event network-vif-unplugged-0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-28bb7ed2-9ead-4393-9f65-b8638546b0f8 req-238a23f8-e1a8-4d08-baa5-0cc430e7f8a6 service nova] Acquiring lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-28bb7ed2-9ead-4393-9f65-b8638546b0f8 req-238a23f8-e1a8-4d08-baa5-0cc430e7f8a6 service nova] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-28bb7ed2-9ead-4393-9f65-b8638546b0f8 req-238a23f8-e1a8-4d08-baa5-0cc430e7f8a6 service nova] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-28bb7ed2-9ead-4393-9f65-b8638546b0f8 req-238a23f8-e1a8-4d08-baa5-0cc430e7f8a6 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] No waiting events found dispatching network-vif-unplugged-0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-28bb7ed2-9ead-4393-9f65-b8638546b0f8 req-238a23f8-e1a8-4d08-baa5-0cc430e7f8a6 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Received event network-vif-unplugged-0b301be3-2fd0-4e29-9740-a2ce2193d477 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Updating instance_info_cache with network_info: [{"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:33:35 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Instance destroyed successfully.
Apr 14 04:33:35 user nova-compute[71317]: DEBUG nova.objects.instance [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lazy-loading 'resources' on Instance uuid af7befec-3f77-4eb7-be53-d1b1e791713e {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:28:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-2095931005',display_name='tempest-TestVolumeBootPattern-volume-backed-server-2095931005',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-2095931005',id=30,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:29:06Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-bma2a5v4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:29:07Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=af7befec-3f77-4eb7-be53-d1b1e791713e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "address": "fa:16:3e:ff:82:75", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0b301be3-2f", "ovs_interfaceid": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "address": "fa:16:3e:ff:82:75", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0b301be3-2f", "ovs_interfaceid": "0b301be3-2fd0-4e29-9740-a2ce2193d477", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ff:82:75,bridge_name='br-int',has_traffic_filtering=True,id=0b301be3-2fd0-4e29-9740-a2ce2193d477,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0b301be3-2f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_vif [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ff:82:75,bridge_name='br-int',has_traffic_filtering=True,id=0b301be3-2fd0-4e29-9740-a2ce2193d477,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0b301be3-2f') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap0b301be3-2f, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:33:35 user nova-compute[71317]: INFO os_vif [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ff:82:75,bridge_name='br-int',has_traffic_filtering=True,id=0b301be3-2fd0-4e29-9740-a2ce2193d477,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0b301be3-2f')
Apr 14 04:33:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'myaLbCEo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:33:35 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f392914a-3e25-491c-a579-261b9e4b2fea]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:33:35 user nova-compute[71317]: No targets found.
Apr 14 04:33:35 user nova-compute[71317]: iSNS:
Apr 14 04:33:35 user nova-compute[71317]: No targets found.
Apr 14 04:33:35 user nova-compute[71317]: STATIC:
Apr 14 04:33:35 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269
Apr 14 04:33:35 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:33:35 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:33:35 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970
Apr 14 04:33:35 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:33:35 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:33:35 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895
Apr 14 04:33:35 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:33:35 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:33:35 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:33:35 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:33:35 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:33:35 user nova-compute[71317]: FIRMWARE:
Apr 14 04:33:35 user nova-compute[71317]: No targets found.
Apr 14 04:33:35 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:33:35 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:33:35 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[89e67707-4a58-481b-8261-5196c433b82b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[930cf20c-da39-455c-b6da-14b62c4b5441]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:33:35 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:33:35 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:33:35 user nova-compute[71317]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:33:35 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:33:35 user nova-compute[71317]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 (non-flash)
Apr 14 04:33:35 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:33:35 user nova-compute[71317]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:33:35 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c41e6320>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895'): ({'sde'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Removing single pathed devices sde {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2090e466-1d0e-47e9-b0e0-ce12fe28f1b2]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec180278-72dd-4007-b174-020f25f0c3e5]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flushing IO for device /dev/sde {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4efbd630-f0b9-48fa-82a3-29b63ce6a96b]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.084s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[47df2a57-958f-4868-a6cf-4848fc2655ed]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[709c9249-68d0-4b28-b388-735294305e44]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:35 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --logout" returned: 0 in 0.056s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c55599d1-26c8-4671-be0d-b56272aafca5]: (4, ('Logging out of session [sid: 38, target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895, portal: 172.16.0.220,3260]\nLogout of [sid: 38, target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 38, target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895, portal: 172.16.0.220,3260]
Apr 14 04:33:36 user nova-compute[71317]: Logout of [sid: 38, target: iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895, portal: 172.16.0.220,3260] successful.
Apr 14 04:33:36 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:300999d2-cadf-4299-8a05-85dec6b7d895 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3214d6e-2c19-4dc2-a33a-08fa8b26b01a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.380s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== disconnect_volume: return (381ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:33:36 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Deleting instance files /opt/stack/data/nova/instances/af7befec-3f77-4eb7-be53-d1b1e791713e_del
Apr 14 04:33:36 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Deletion of /opt/stack/data/nova/instances/af7befec-3f77-4eb7-be53-d1b1e791713e_del complete
Apr 14 04:33:36 user nova-compute[71317]: INFO nova.compute.manager [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Took 1.24 seconds to destroy the instance on the hypervisor.
Apr 14 04:33:36 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:33:36 user nova-compute[71317]: DEBUG nova.compute.manager [req-c4c5090c-80b9-467c-9fb5-5db6c6f578f5 req-7c17e9e0-e460-4cce-a986-c85c4a98dc47 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Received event network-vif-deleted-0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:36 user nova-compute[71317]: INFO nova.compute.manager [req-c4c5090c-80b9-467c-9fb5-5db6c6f578f5 req-7c17e9e0-e460-4cce-a986-c85c4a98dc47 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Neutron deleted interface 0b301be3-2fd0-4e29-9740-a2ce2193d477; detaching it from the instance and deleting it from the info cache
Apr 14 04:33:36 user nova-compute[71317]: DEBUG nova.network.neutron [req-c4c5090c-80b9-467c-9fb5-5db6c6f578f5 req-7c17e9e0-e460-4cce-a986-c85c4a98dc47 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:33:36 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Took 0.51 seconds to deallocate network for instance.
Apr 14 04:33:36 user nova-compute[71317]: DEBUG nova.compute.manager [req-c4c5090c-80b9-467c-9fb5-5db6c6f578f5 req-7c17e9e0-e460-4cce-a986-c85c4a98dc47 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Detach interface failed, port_id=0b301be3-2fd0-4e29-9740-a2ce2193d477, reason: Instance af7befec-3f77-4eb7-be53-d1b1e791713e could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:33:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-a544d41c-3e1b-411d-af88-e122d22331a0 req-cce5f959-c4dd-4744-82de-fff584497451 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Received event network-vif-plugged-0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a544d41c-3e1b-411d-af88-e122d22331a0 req-cce5f959-c4dd-4744-82de-fff584497451 service nova] Acquiring lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a544d41c-3e1b-411d-af88-e122d22331a0 req-cce5f959-c4dd-4744-82de-fff584497451 service nova] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a544d41c-3e1b-411d-af88-e122d22331a0 req-cce5f959-c4dd-4744-82de-fff584497451 service nova] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:37 user nova-compute[71317]: DEBUG nova.compute.manager [req-a544d41c-3e1b-411d-af88-e122d22331a0 req-cce5f959-c4dd-4744-82de-fff584497451 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] No waiting events found dispatching network-vif-plugged-0b301be3-2fd0-4e29-9740-a2ce2193d477 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:33:37 user nova-compute[71317]: WARNING nova.compute.manager [req-a544d41c-3e1b-411d-af88-e122d22331a0 req-cce5f959-c4dd-4744-82de-fff584497451 service nova] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Received unexpected event network-vif-plugged-0b301be3-2fd0-4e29-9740-a2ce2193d477 for instance with vm_state active and task_state deleting.
Apr 14 04:33:37 user nova-compute[71317]: INFO nova.compute.manager [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Took 0.84 seconds to detach 1 volumes for instance.
Apr 14 04:33:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Deleting volume: 300999d2-cadf-4299-8a05-85dec6b7d895 {{(pid=71317) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 14 04:33:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:37 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.412s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:38 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Deleted allocations for instance af7befec-3f77-4eb7-be53-d1b1e791713e
Apr 14 04:33:38 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-885a1967-e3a5-4703-b8d4-3086631b0d12 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "af7befec-3f77-4eb7-be53-d1b1e791713e" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.270s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:38 user nova-compute[71317]: INFO nova.compute.manager [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Terminating instance
Apr 14 04:33:38 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-5dd94d81-6bd1-485e-9582-bfe2c90ef9e4 req-48260d59-3f81-43e8-a9ce-af61f422c87d service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-unplugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5dd94d81-6bd1-485e-9582-bfe2c90ef9e4 req-48260d59-3f81-43e8-a9ce-af61f422c87d service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5dd94d81-6bd1-485e-9582-bfe2c90ef9e4 req-48260d59-3f81-43e8-a9ce-af61f422c87d service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5dd94d81-6bd1-485e-9582-bfe2c90ef9e4 req-48260d59-3f81-43e8-a9ce-af61f422c87d service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-5dd94d81-6bd1-485e-9582-bfe2c90ef9e4 req-48260d59-3f81-43e8-a9ce-af61f422c87d service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-unplugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:33:38 user nova-compute[71317]: DEBUG nova.compute.manager [req-5dd94d81-6bd1-485e-9582-bfe2c90ef9e4 req-48260d59-3f81-43e8-a9ce-af61f422c87d service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-unplugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:33:39 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Instance destroyed successfully.
Apr 14 04:33:39 user nova-compute[71317]: DEBUG nova.objects.instance [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lazy-loading 'resources' on Instance uuid 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:33:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:16:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1529931408',display_name='tempest-ServerRescueNegativeTestJSON-server-1529931408',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1529931408',id=5,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:21:35Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='33320dc2f2c14ecfa5abf6574029c36c',ramdisk_id='',reservation_id='r-wldnu0b2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-508041341',owner_user_name='tempest-ServerRescueNegativeTestJSON-508041341-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:23:25Z,user_data=None,user_id='5234914aec8e443b8a38ac6b38729031',uuid=586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:33:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converting VIF {"id": "593a3298-746a-48af-96dc-4c572e0d59c2", "address": "fa:16:3e:fb:7a:65", "network": {"id": "b413a761-e152-48ff-a963-b8e2cbab4a86", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1142082540-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "33320dc2f2c14ecfa5abf6574029c36c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap593a3298-74", "ovs_interfaceid": "593a3298-746a-48af-96dc-4c572e0d59c2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:33:39 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:fb:7a:65,bridge_name='br-int',has_traffic_filtering=True,id=593a3298-746a-48af-96dc-4c572e0d59c2,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap593a3298-74') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:33:39 user nova-compute[71317]: DEBUG os_vif [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:fb:7a:65,bridge_name='br-int',has_traffic_filtering=True,id=593a3298-746a-48af-96dc-4c572e0d59c2,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap593a3298-74') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:33:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap593a3298-74, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:33:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:39 user nova-compute[71317]: INFO os_vif [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:fb:7a:65,bridge_name='br-int',has_traffic_filtering=True,id=593a3298-746a-48af-96dc-4c572e0d59c2,network=Network(b413a761-e152-48ff-a963-b8e2cbab4a86),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap593a3298-74')
Apr 14 04:33:39 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Deleting instance files /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc_del
Apr 14 04:33:39 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Deletion of /opt/stack/data/nova/instances/586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc_del complete
Apr 14 04:33:39 user nova-compute[71317]: INFO nova.compute.manager [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Took 0.98 seconds to destroy the instance on the hypervisor.
Apr 14 04:33:39 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:33:39 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:33:39 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:33:39 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:33:39 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Took 0.53 seconds to deallocate network for instance.
Apr 14 04:33:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.333s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.154s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:33:40 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Deleted allocations for instance 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e39b4277-22aa-4219-b1dc-faf2726caec6 tempest-ServerRescueNegativeTestJSON-508041341 tempest-ServerRescueNegativeTestJSON-508041341-project-member] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.035s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-0c092958-1d4c-46a8-9d1e-d0bff2b8a812 req-9791eec4-a225-4508-a088-727ffe7282fd service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0c092958-1d4c-46a8-9d1e-d0bff2b8a812 req-9791eec4-a225-4508-a088-727ffe7282fd service nova] Acquiring lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0c092958-1d4c-46a8-9d1e-d0bff2b8a812 req-9791eec4-a225-4508-a088-727ffe7282fd service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0c092958-1d4c-46a8-9d1e-d0bff2b8a812 req-9791eec4-a225-4508-a088-727ffe7282fd service nova] Lock "586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-0c092958-1d4c-46a8-9d1e-d0bff2b8a812 req-9791eec4-a225-4508-a088-727ffe7282fd service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] No waiting events found dispatching network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:33:40 user nova-compute[71317]: WARNING nova.compute.manager [req-0c092958-1d4c-46a8-9d1e-d0bff2b8a812 req-9791eec4-a225-4508-a088-727ffe7282fd service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received unexpected event network-vif-plugged-593a3298-746a-48af-96dc-4c572e0d59c2 for instance with vm_state deleted and task_state None.
Apr 14 04:33:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-0c092958-1d4c-46a8-9d1e-d0bff2b8a812 req-9791eec4-a225-4508-a088-727ffe7282fd service nova] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Received event network-vif-deleted-593a3298-746a-48af-96dc-4c572e0d59c2 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:33:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:33:42 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:33:42 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7927MB free_disk=26.195632934570312GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 19a166de-f0c9-43f2-8c88-585e0d5d22cc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 390ca845-5660-49d0-bd8a-fcfc39176ca5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 970e81d2-c227-4eab-92dd-1ed732e67968 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 695d931b-1f30-4d57-b6d7-d199d2874e65 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3746b520-c10a-413e-9f1f-8807b3fd7c9a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 8 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1536MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=8 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:33:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.386s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:33:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:43 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:33:43 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:33:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:33:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:50 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446815.6547167, af7befec-3f77-4eb7-be53-d1b1e791713e => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:33:50 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] VM Stopped (Lifecycle Event)
Apr 14 04:33:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-aace6b41-3fd5-4a9d-883c-c2348386c374 None None] [instance: af7befec-3f77-4eb7-be53-d1b1e791713e] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:54 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446819.200799, 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:33:54 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] VM Stopped (Lifecycle Event)
Apr 14 04:33:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bc8625c3-4850-4307-8156-9e29f417ab5a None None] [instance: 586ca3d5-9ac5-4d3f-868d-7ff7aedf5fbc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:33:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:33:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:33:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:34:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "4b704133-1b1a-4579-857a-c74cd8b24fae" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG nova.compute.manager [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:34:33 user nova-compute[71317]: INFO nova.compute.claims [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Claim successful on node user
Apr 14 04:34:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.496s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG nova.compute.manager [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG nova.compute.manager [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:34:33 user nova-compute[71317]: DEBUG nova.network.neutron [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:34:33 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:34:33 user nova-compute[71317]: DEBUG nova.compute.manager [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:34:34 user nova-compute[71317]: INFO nova.virt.block_device [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Booting with volume 495f173c-6873-42b6-a0fe-42da67f3aa75 at /dev/vda
Apr 14 04:34:34 user nova-compute[71317]: DEBUG nova.policy [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bcf79de7da27495dbfd14c556b5b54c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '693d95858c624feaa60114aa89082186', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG os_brick.utils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[30f130a1-19bd-4e34-9561-6b5215a8ed9c]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f8fe153-f59e-4173-a0d9-e2efc3cf40a4]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[099665ba-c85a-40a3-ac21-4071b1138766]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a126e65a-cb02-4cf2-8533-36234c3e6bd4]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:34:34 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[738d31b0-e5e7-4232-8e43-86d0032b1be1]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG os_brick.utils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== get_connector_properties: return (181ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Updating existing volume attachment record: 747d47a5-c9e1-4781-a1f4-f44c95772ca8 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:34:34 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:34:35 user nova-compute[71317]: DEBUG nova.network.neutron [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Successfully created port: c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:34:35 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updating instance_info_cache with network_info: [{"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:34:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:34:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG nova.network.neutron [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Successfully updated port: c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "refresh_cache-4b704133-1b1a-4579-857a-c74cd8b24fae" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquired lock "refresh_cache-4b704133-1b1a-4579-857a-c74cd8b24fae" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG nova.network.neutron [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG nova.compute.manager [req-98bce72e-cb54-4fbf-ae3d-5b066f7652f2 req-c35942f5-b7f4-4ac8-87d4-454267c0eefe service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Received event network-changed-c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG nova.compute.manager [req-98bce72e-cb54-4fbf-ae3d-5b066f7652f2 req-c35942f5-b7f4-4ac8-87d4-454267c0eefe service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Refreshing instance network info cache due to event network-changed-c0df2242-82c7-408b-bdc2-ab608dee7c1a. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-98bce72e-cb54-4fbf-ae3d-5b066f7652f2 req-c35942f5-b7f4-4ac8-87d4-454267c0eefe service nova] Acquiring lock "refresh_cache-4b704133-1b1a-4579-857a-c74cd8b24fae" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG nova.network.neutron [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG nova.network.neutron [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Updating instance_info_cache with network_info: [{"id": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "address": "fa:16:3e:6d:f7:af", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0df2242-82", "ovs_interfaceid": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Releasing lock "refresh_cache-4b704133-1b1a-4579-857a-c74cd8b24fae" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG nova.compute.manager [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Instance network_info: |[{"id": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "address": "fa:16:3e:6d:f7:af", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0df2242-82", "ovs_interfaceid": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-98bce72e-cb54-4fbf-ae3d-5b066f7652f2 req-c35942f5-b7f4-4ac8-87d4-454267c0eefe service nova] Acquired lock "refresh_cache-4b704133-1b1a-4579-857a-c74cd8b24fae" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG nova.network.neutron [req-98bce72e-cb54-4fbf-ae3d-5b066f7652f2 req-c35942f5-b7f4-4ac8-87d4-454267c0eefe service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Refreshing network info cache for port c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG nova.network.neutron [req-98bce72e-cb54-4fbf-ae3d-5b066f7652f2 req-c35942f5-b7f4-4ac8-87d4-454267c0eefe service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Updated VIF entry in instance network info cache for port c0df2242-82c7-408b-bdc2-ab608dee7c1a. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG nova.network.neutron [req-98bce72e-cb54-4fbf-ae3d-5b066f7652f2 req-c35942f5-b7f4-4ac8-87d4-454267c0eefe service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Updating instance_info_cache with network_info: [{"id": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "address": "fa:16:3e:6d:f7:af", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0df2242-82", "ovs_interfaceid": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:34:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-98bce72e-cb54-4fbf-ae3d-5b066f7652f2 req-c35942f5-b7f4-4ac8-87d4-454267c0eefe service nova] Releasing lock "refresh_cache-4b704133-1b1a-4579-857a-c74cd8b24fae" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:34:37 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:34:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:34:39 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Creating image(s)
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Ensure instance console log exists: /opt/stack/data/nova/instances/4b704133-1b1a-4579-857a-c74cd8b24fae/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Start _get_guest_xml network_info=[{"id": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "address": "fa:16:3e:6d:f7:af", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0df2242-82", "ovs_interfaceid": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tLxSpwaP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '4b704133-1b1a-4579-857a-c74cd8b24fae', 'attached_at': '', 'detached_at': '', 'volume_id': '495f173c-6873-42b6-a0fe-42da67f3aa75', 'serial': '495f173c-6873-42b6-a0fe-42da67f3aa75'}, 'attachment_id': '747d47a5-c9e1-4781-a1f4-f44c95772ca8', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:34:39 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:34:39 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tLxSpwaP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:34:39 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:34:39 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:34:39 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[93db9a03-7053-4972-ad08-f86bec7ee44c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:34:39 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[74996125-a8c6-4b9b-874b-4cab7750b7d1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75] added
Apr 14 04:34:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5018944-317b-4574-b6e2-a00e8de6913e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a61bcf47-9a5b-4e81-888f-94890fecbc21]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tLxSpwaP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tLxSpwaP" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b864d774-6462-4c8d-a447-251fb1f9ad30]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'tLxSpwaP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[529dc04f-6ec9-4aaf-9e9f-127d6abba1bc]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5956436b-5244-49f5-8267-fedb90e9f06f]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --login" returned: 0 in 0.067s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9801dedb-7654-41bd-aba3-fc3524c85686]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260]
Apr 14 04:34:39 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260] successful.
Apr 14 04:34:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7d624f5-d34b-406e-8fbe-f4305415e788]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[62f7fed2-e43a-4880-94c0-f4e11e0af37a]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:34:39 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.319s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] HCTL ('36', '-', '-', 0) found on session 46 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[349c0e9f-02a1-42d3-a425-eec619bbee55]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Searching for a device in session 46 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tLxSpwaP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:34:39 user nova-compute[71317]: INFO nova.compute.manager [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Rescuing
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "refresh_cache-970e81d2-c227-4eab-92dd-1ed732e67968" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquired lock "refresh_cache-970e81d2-c227-4eab-92dd-1ed732e67968" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:34:39 user nova-compute[71317]: DEBUG nova.network.neutron [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.360s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== connect_volume: return (1361ms) {'type': 'block', 'scsi_wwn': '23132643462623363', 'path': '/dev/sde'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23132643462623363', 'path': '/dev/sde'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:34:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1686619211',display_name='tempest-TestVolumeBootPattern-server-1686619211',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1686619211',id=38,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFZO4o4kQgpEvaFt8iKOk4kfW7YF9jZ6XS2jVS2UAmaxmR76rP/jf7a3pf4xOyQu5V8HIaLR//SNI2rVtnFikRInNPtRWTjIBvrk9d4sXMIgR6FP64tMb+FLBXcO2su56A==',key_name='tempest-TestVolumeBootPattern-1640497739',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-bxnjue41',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:34:34Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=4b704133-1b1a-4579-857a-c74cd8b24fae,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "address": "fa:16:3e:6d:f7:af", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0df2242-82", "ovs_interfaceid": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "address": "fa:16:3e:6d:f7:af", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0df2242-82", "ovs_interfaceid": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6d:f7:af,bridge_name='br-int',has_traffic_filtering=True,id=c0df2242-82c7-408b-bdc2-ab608dee7c1a,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0df2242-82') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG nova.objects.instance [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lazy-loading 'pci_devices' on Instance uuid 4b704133-1b1a-4579-857a-c74cd8b24fae {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:34:40 user nova-compute[71317]:   <uuid>4b704133-1b1a-4579-857a-c74cd8b24fae</uuid>
Apr 14 04:34:40 user nova-compute[71317]:   <name>instance-00000026</name>
Apr 14 04:34:40 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:34:40 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:34:40 user nova-compute[71317]:   <metadata>
Apr 14 04:34:40 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:34:40 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:34:40 user nova-compute[71317]:       <nova:name>tempest-TestVolumeBootPattern-server-1686619211</nova:name>
Apr 14 04:34:40 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:34:39</nova:creationTime>
Apr 14 04:34:40 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:34:40 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:34:40 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:34:40 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:34:40 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:34:40 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:34:40 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:34:40 user nova-compute[71317]:       <nova:owner>
Apr 14 04:34:40 user nova-compute[71317]:         <nova:user uuid="bcf79de7da27495dbfd14c556b5b54c7">tempest-TestVolumeBootPattern-1780160917-project-member</nova:user>
Apr 14 04:34:40 user nova-compute[71317]:         <nova:project uuid="693d95858c624feaa60114aa89082186">tempest-TestVolumeBootPattern-1780160917</nova:project>
Apr 14 04:34:40 user nova-compute[71317]:       </nova:owner>
Apr 14 04:34:40 user nova-compute[71317]:       <nova:ports>
Apr 14 04:34:40 user nova-compute[71317]:         <nova:port uuid="c0df2242-82c7-408b-bdc2-ab608dee7c1a">
Apr 14 04:34:40 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 04:34:40 user nova-compute[71317]:         </nova:port>
Apr 14 04:34:40 user nova-compute[71317]:       </nova:ports>
Apr 14 04:34:40 user nova-compute[71317]:     </nova:instance>
Apr 14 04:34:40 user nova-compute[71317]:   </metadata>
Apr 14 04:34:40 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:34:40 user nova-compute[71317]:     <system>
Apr 14 04:34:40 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:34:40 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:34:40 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:34:40 user nova-compute[71317]:       <entry name="serial">4b704133-1b1a-4579-857a-c74cd8b24fae</entry>
Apr 14 04:34:40 user nova-compute[71317]:       <entry name="uuid">4b704133-1b1a-4579-857a-c74cd8b24fae</entry>
Apr 14 04:34:40 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:34:40 user nova-compute[71317]:     </system>
Apr 14 04:34:40 user nova-compute[71317]:   </sysinfo>
Apr 14 04:34:40 user nova-compute[71317]:   <os>
Apr 14 04:34:40 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:34:40 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:34:40 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:34:40 user nova-compute[71317]:   </os>
Apr 14 04:34:40 user nova-compute[71317]:   <features>
Apr 14 04:34:40 user nova-compute[71317]:     <acpi/>
Apr 14 04:34:40 user nova-compute[71317]:     <apic/>
Apr 14 04:34:40 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:34:40 user nova-compute[71317]:   </features>
Apr 14 04:34:40 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:34:40 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:34:40 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:34:40 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:34:40 user nova-compute[71317]:   </clock>
Apr 14 04:34:40 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:34:40 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:34:40 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:34:40 user nova-compute[71317]:   </cpu>
Apr 14 04:34:40 user nova-compute[71317]:   <devices>
Apr 14 04:34:40 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:34:40 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:34:40 user nova-compute[71317]:       <source dev="/dev/sde"/>
Apr 14 04:34:40 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:34:40 user nova-compute[71317]:       <serial>495f173c-6873-42b6-a0fe-42da67f3aa75</serial>
Apr 14 04:34:40 user nova-compute[71317]:     </disk>
Apr 14 04:34:40 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:34:40 user nova-compute[71317]:       <mac address="fa:16:3e:6d:f7:af"/>
Apr 14 04:34:40 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:34:40 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:34:40 user nova-compute[71317]:       <target dev="tapc0df2242-82"/>
Apr 14 04:34:40 user nova-compute[71317]:     </interface>
Apr 14 04:34:40 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:34:40 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/4b704133-1b1a-4579-857a-c74cd8b24fae/console.log" append="off"/>
Apr 14 04:34:40 user nova-compute[71317]:     </serial>
Apr 14 04:34:40 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:34:40 user nova-compute[71317]:     <video>
Apr 14 04:34:40 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:34:40 user nova-compute[71317]:     </video>
Apr 14 04:34:40 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:34:40 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:34:40 user nova-compute[71317]:     </rng>
Apr 14 04:34:40 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:34:40 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:34:40 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:34:40 user nova-compute[71317]:     </memballoon>
Apr 14 04:34:40 user nova-compute[71317]:   </devices>
Apr 14 04:34:40 user nova-compute[71317]: </domain>
Apr 14 04:34:40 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:34:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1686619211',display_name='tempest-TestVolumeBootPattern-server-1686619211',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1686619211',id=38,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFZO4o4kQgpEvaFt8iKOk4kfW7YF9jZ6XS2jVS2UAmaxmR76rP/jf7a3pf4xOyQu5V8HIaLR//SNI2rVtnFikRInNPtRWTjIBvrk9d4sXMIgR6FP64tMb+FLBXcO2su56A==',key_name='tempest-TestVolumeBootPattern-1640497739',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-bxnjue41',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:34:34Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=4b704133-1b1a-4579-857a-c74cd8b24fae,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "address": "fa:16:3e:6d:f7:af", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0df2242-82", "ovs_interfaceid": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "address": "fa:16:3e:6d:f7:af", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0df2242-82", "ovs_interfaceid": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6d:f7:af,bridge_name='br-int',has_traffic_filtering=True,id=c0df2242-82c7-408b-bdc2-ab608dee7c1a,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0df2242-82') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG os_vif [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6d:f7:af,bridge_name='br-int',has_traffic_filtering=True,id=c0df2242-82c7-408b-bdc2-ab608dee7c1a,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0df2242-82') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG nova.network.neutron [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Updating instance_info_cache with network_info: [{"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc0df2242-82, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc0df2242-82, col_values=(('external_ids', {'iface-id': 'c0df2242-82c7-408b-bdc2-ab608dee7c1a', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6d:f7:af', 'vm-uuid': '4b704133-1b1a-4579-857a-c74cd8b24fae'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Releasing lock "refresh_cache-970e81d2-c227-4eab-92dd-1ed732e67968" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:40 user nova-compute[71317]: INFO os_vif [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6d:f7:af,bridge_name='br-int',has_traffic_filtering=True,id=c0df2242-82c7-408b-bdc2-ab608dee7c1a,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0df2242-82')
Apr 14 04:34:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] No VIF found with MAC fa:16:3e:6d:f7:af, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:34:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-7caa8efe-336f-464f-84ee-6c7b06af8fcb req-5d9f1ba0-e806-47d7-a481-6136d8d24b27 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-unplugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:34:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7caa8efe-336f-464f-84ee-6c7b06af8fcb req-5d9f1ba0-e806-47d7-a481-6136d8d24b27 service nova] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7caa8efe-336f-464f-84ee-6c7b06af8fcb req-5d9f1ba0-e806-47d7-a481-6136d8d24b27 service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7caa8efe-336f-464f-84ee-6c7b06af8fcb req-5d9f1ba0-e806-47d7-a481-6136d8d24b27 service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-7caa8efe-336f-464f-84ee-6c7b06af8fcb req-5d9f1ba0-e806-47d7-a481-6136d8d24b27 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] No waiting events found dispatching network-vif-unplugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:34:41 user nova-compute[71317]: WARNING nova.compute.manager [req-7caa8efe-336f-464f-84ee-6c7b06af8fcb req-5d9f1ba0-e806-47d7-a481-6136d8d24b27 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received unexpected event network-vif-unplugged-45a5a643-510a-45e2-a8e4-12eb53631daf for instance with vm_state active and task_state rescuing.
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:34:42 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Instance destroyed successfully.
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:34:42 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Attempting a stable device rescue
Apr 14 04:34:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=71317) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Instance directory exists: not creating {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 04:34:42 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Creating image(s)
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "/opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "/opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "/opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.008s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG nova.objects.instance [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'trusted_certs' on Instance uuid 970e81d2-c227-4eab-92dd-1ed732e67968 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "c262e1265bd42d036e70ac0a04884f55c3aa3f91" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "c262e1265bd42d036e70ac0a04884f55c3aa3f91" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91.part --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-d33c8da1-2ec3-4ede-99c8-889fabbe1a5d req-e0dfd4b1-3066-40cc-b1d8-0d0eca970904 service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Received event network-vif-plugged-c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d33c8da1-2ec3-4ede-99c8-889fabbe1a5d req-e0dfd4b1-3066-40cc-b1d8-0d0eca970904 service nova] Acquiring lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d33c8da1-2ec3-4ede-99c8-889fabbe1a5d req-e0dfd4b1-3066-40cc-b1d8-0d0eca970904 service nova] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d33c8da1-2ec3-4ede-99c8-889fabbe1a5d req-e0dfd4b1-3066-40cc-b1d8-0d0eca970904 service nova] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-d33c8da1-2ec3-4ede-99c8-889fabbe1a5d req-e0dfd4b1-3066-40cc-b1d8-0d0eca970904 service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] No waiting events found dispatching network-vif-plugged-c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:34:42 user nova-compute[71317]: WARNING nova.compute.manager [req-d33c8da1-2ec3-4ede-99c8-889fabbe1a5d req-e0dfd4b1-3066-40cc-b1d8-0d0eca970904 service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Received unexpected event network-vif-plugged-c0df2242-82c7-408b-bdc2-ab608dee7c1a for instance with vm_state building and task_state spawning.
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91.part --force-share --output=json" returned: 0 in 0.172s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG nova.virt.images [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] 9ee24f9a-817e-4cc1-b650-403209dbb85d was qcow2, converting to raw {{(pid=71317) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91.part /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91.converted {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91.part /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91.converted" returned: 0 in 0.475s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91.converted --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-ba9d0115-9d36-4a6d-a82d-bfccb9641380 req-e05bf042-7d97-4669-90f8-ffe2e6861be8 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ba9d0115-9d36-4a6d-a82d-bfccb9641380 req-e05bf042-7d97-4669-90f8-ffe2e6861be8 service nova] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ba9d0115-9d36-4a6d-a82d-bfccb9641380 req-e05bf042-7d97-4669-90f8-ffe2e6861be8 service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ba9d0115-9d36-4a6d-a82d-bfccb9641380 req-e05bf042-7d97-4669-90f8-ffe2e6861be8 service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-ba9d0115-9d36-4a6d-a82d-bfccb9641380 req-e05bf042-7d97-4669-90f8-ffe2e6861be8 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] No waiting events found dispatching network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:34:43 user nova-compute[71317]: WARNING nova.compute.manager [req-ba9d0115-9d36-4a6d-a82d-bfccb9641380 req-e05bf042-7d97-4669-90f8-ffe2e6861be8 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received unexpected event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf for instance with vm_state active and task_state rescuing.
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91.converted --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "c262e1265bd42d036e70ac0a04884f55c3aa3f91" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.495s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "c262e1265bd42d036e70ac0a04884f55c3aa3f91" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "c262e1265bd42d036e70ac0a04884f55c3aa3f91" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91 --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91,backing_fmt=raw /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.rescue {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91,backing_fmt=raw /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.rescue" returned: 0 in 0.047s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "c262e1265bd42d036e70ac0a04884f55c3aa3f91" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.194s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG nova.objects.instance [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'migration_context' on Instance uuid 970e81d2-c227-4eab-92dd-1ed732e67968 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Start _get_guest_xml network_info=[{"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "vif_mac": "fa:16:3e:cd:43:7e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '9ee24f9a-817e-4cc1-b650-403209dbb85d', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'deafxyMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, 'status': 'reserved', 'instance': '970e81d2-c227-4eab-92dd-1ed732e67968', 'attached_at': '', 'detached_at': '', 'volume_id': '9108333c-dbbf-4edb-a2b5-c6f034073970', 'serial': '9108333c-dbbf-4edb-a2b5-c6f034073970'}, 'attachment_id': '5ae6bc76-75a1-4f44-99a1-fdb9bf96b0ef', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG nova.objects.instance [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'resources' on Instance uuid 970e81d2-c227-4eab-92dd-1ed732e67968 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:34:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.objects.instance [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'numa_topology' on Instance uuid 970e81d2-c227-4eab-92dd-1ed732e67968 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:34:44 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:34:44 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.objects.instance [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'vcpu_model' on Instance uuid 970e81d2-c227-4eab-92dd-1ed732e67968 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'deafxyMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:34:44 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:34:44 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:34:44 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[72a17d83-2003-445c-bce7-93b227ba6b3a]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = deafxyMo\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Apr 14 04:34:44 user nova-compute[71317]: node.name = iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970
Apr 14 04:34:44 user nova-compute[71317]: node.tpgt = -1
Apr 14 04:34:44 user nova-compute[71317]: node.startup = automatic
Apr 14 04:34:44 user nova-compute[71317]: node.leading_login = No
Apr 14 04:34:44 user nova-compute[71317]: iface.iscsi_ifacename = default
Apr 14 04:34:44 user nova-compute[71317]: iface.net_ifacename = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.ipaddress = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.prefix_len = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.hwaddress = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.transport_name = tcp
Apr 14 04:34:44 user nova-compute[71317]: iface.initiatorname = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.state = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.vlan_id = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.vlan_priority = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.vlan_state = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.iface_num = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.mtu = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.port = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.bootproto = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.subnet_mask = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.gateway = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.dhcp_alt_client_id_state = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.dhcp_alt_client_id = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.dhcp_dns = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.dhcp_learn_iqn = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.dhcp_req_vendor_id_state = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.dhcp_vendor_id_state = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.dhcp_vendor_id = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.dhcp_slp_da = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.fragmentation = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.gratuitous_arp = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.incoming_forwarding = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.tos_state = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.tos = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.ttl = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.delayed_ack = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.tcp_nagle = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.tcp_wsf_state = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.tcp_wsf = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.tcp_timer_scale = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.tcp_timestamp = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.redirect = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.def_task_mgmt_timeout = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.header_digest = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.data_digest = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.immediate_data = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.initial_r2t = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.data_seq_inorder = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.data_pdu_inorder = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.erl = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.max_receive_data_len = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.first_burst_len = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.max_outstanding_r2t = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.max_burst_len = 0
Apr 14 04:34:44 user nova-compute[71317]: iface.chap_auth = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.bidi_chap = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.strict_login_compliance = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.discovery_auth = <empty>
Apr 14 04:34:44 user nova-compute[71317]: iface.discovery_logout = <empty>
Apr 14 04:34:44 user nova-compute[71317]: node.discovery_address = <empty>
Apr 14 04:34:44 user nova-compute[71317]: node.discovery_port = 0
Apr 14 04:34:44 user nova-compute[71317]: node.discovery_type = static
Apr 14 04:34:44 user nova-compute[71317]: node.session.initial_cmdsn = 0
Apr 14 04:34:44 user nova-compute[71317]: node.session.initial_login_retry_max = 8
Apr 14 04:34:44 user nova-compute[71317]: node.session.xmit_thread_priority = -20
Apr 14 04:34:44 user nova-compute[71317]: node.session.cmds_max = 128
Apr 14 04:34:44 user nova-compute[71317]: node.session.queue_depth = 32
Apr 14 04:34:44 user nova-compute[71317]: node.session.nr_sessions = 1
Apr 14 04:34:44 user nova-compute[71317]: node.session.auth.authmethod = CHAP
Apr 14 04:34:44 user nova-compute[71317]: node.session.auth.username = deafxyMo
Apr 14 04:34:44 user nova-compute[71317]: node.session.auth.password = ***
Apr 14 04:34:44 user nova-compute[71317]: node.session.auth.username_in = <empty>
Apr 14 04:34:44 user nova-compute[71317]: node.session.auth.password_in = <empty>
Apr 14 04:34:44 user nova-compute[71317]: node.session.auth.chap_algs = MD5
Apr 14 04:34:44 user nova-compute[71317]: node.session.timeo.replacement_timeout = 120
Apr 14 04:34:44 user nova-compute[71317]: node.session.err_timeo.abort_timeout = 15
Apr 14 04:34:44 user nova-compute[71317]: node.session.err_timeo.lu_reset_timeout = 30
Apr 14 04:34:44 user nova-compute[71317]: node.session.err_timeo.tgt_reset_timeout = 30
Apr 14 04:34:44 user nova-compute[71317]: node.session.err_timeo.host_reset_timeout = 60
Apr 14 04:34:44 user nova-compute[71317]: node.session.iscsi.FastAbort = Yes
Apr 14 04:34:44 user nova-compute[71317]: node.session.iscsi.InitialR2T = No
Apr 14 04:34:44 user nova-compute[71317]: node.session.iscsi.ImmediateData = Yes
Apr 14 04:34:44 user nova-compute[71317]: node.session.iscsi.FirstBurstLength = 262144
Apr 14 04:34:44 user nova-compute[71317]: node.session.iscsi.MaxBurstLength = 16776192
Apr 14 04:34:44 user nova-compute[71317]: node.session.iscsi.DefaultTime2Retain = 0
Apr 14 04:34:44 user nova-compute[71317]: node.session.iscsi.DefaultTime2Wait = 2
Apr 14 04:34:44 user nova-compute[71317]: node.session.iscsi.MaxConnections = 1
Apr 14 04:34:44 user nova-compute[71317]: node.session.iscsi.MaxOutstandingR2T = 1
Apr 14 04:34:44 user nova-compute[71317]: node.session.iscsi.ERL = 0
Apr 14 04:34:44 user nova-compute[71317]: node.session.scan = manual
Apr 14 04:34:44 user nova-compute[71317]: node.session.reopen_max = 0
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].address = 172.16.0.220
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].port = 3260
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].startup = manual
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].tcp.window_size = 524288
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].tcp.type_of_service = 0
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].timeo.logout_timeout = 15
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].timeo.login_timeout = 15
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].timeo.auth_timeout = 45
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].timeo.noop_out_interval = 5
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].timeo.noop_out_timeout = 5
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].iscsi.HeaderDigest = None
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].iscsi.DataDigest = None
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].iscsi.IFMarker = No
Apr 14 04:34:44 user nova-compute[71317]: node.conn[0].iscsi.OFMarker = No
Apr 14 04:34:44 user nova-compute[71317]: # END RECORD
Apr 14 04:34:44 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8823f6e1-1dce-4cad-a418-694e6ebca655]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[05d5242c-29d0-4573-9123-ea2bdc42b34a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v deafxyMo {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v deafxyMo" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9eb9f9d1-e0ac-44ed-945d-48d8a7add947]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'deafxyMo'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6998fc1-7611-4ee6-9fd4-fba5d95ff426]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.037s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[df944cbd-df51-4d23-aa01-9a7b6e7f256a]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:34:44 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:34:44 user nova-compute[71317]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:34:44 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:34:44 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:34:44 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:34:44 user nova-compute[71317]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:34:44 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:34:44 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.249s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] HCTL ('34', '0', '0', 0) found on session 44 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Scanning host 34 c: 0, t: 0, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[579c738d-3c38-4a6d-8a05-0b957f830eae]: (4, ('0 0 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Searching for a device in session 44 and hctl ('34', '0', '0', 0) yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'deafxyMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.273s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] <== connect_volume: return (275ms) {'type': 'block', 'scsi_wwn': '26334313830363633', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26334313830363633', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:32:48Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1167245313',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1167245313',id=35,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:33:06Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-djyaua3x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:33:06Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=970e81d2-c227-4eab-92dd-1ed732e67968,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "vif_mac": "fa:16:3e:cd:43:7e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "vif_mac": "fa:16:3e:cd:43:7e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:cd:43:7e,bridge_name='br-int',has_traffic_filtering=True,id=45a5a643-510a-45e2-a8e4-12eb53631daf,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap45a5a643-51') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.objects.instance [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'pci_devices' on Instance uuid 970e81d2-c227-4eab-92dd-1ed732e67968 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:34:44 user nova-compute[71317]:   <uuid>970e81d2-c227-4eab-92dd-1ed732e67968</uuid>
Apr 14 04:34:44 user nova-compute[71317]:   <name>instance-00000023</name>
Apr 14 04:34:44 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:34:44 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:34:44 user nova-compute[71317]:   <metadata>
Apr 14 04:34:44 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:34:44 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:34:44 user nova-compute[71317]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1167245313</nova:name>
Apr 14 04:34:44 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:34:44</nova:creationTime>
Apr 14 04:34:44 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:34:44 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:34:44 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:34:44 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:34:44 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:34:44 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:34:44 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:34:44 user nova-compute[71317]:       <nova:owner>
Apr 14 04:34:44 user nova-compute[71317]:         <nova:user uuid="6ec7103e06e643a6b9b2c908ba8d7b06">tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member</nova:user>
Apr 14 04:34:44 user nova-compute[71317]:         <nova:project uuid="d5995b934b0546c0983387e46e3b3df4">tempest-ServerBootFromVolumeStableRescueTest-247821842</nova:project>
Apr 14 04:34:44 user nova-compute[71317]:       </nova:owner>
Apr 14 04:34:44 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:34:44 user nova-compute[71317]:       <nova:ports>
Apr 14 04:34:44 user nova-compute[71317]:         <nova:port uuid="45a5a643-510a-45e2-a8e4-12eb53631daf">
Apr 14 04:34:44 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 14 04:34:44 user nova-compute[71317]:         </nova:port>
Apr 14 04:34:44 user nova-compute[71317]:       </nova:ports>
Apr 14 04:34:44 user nova-compute[71317]:     </nova:instance>
Apr 14 04:34:44 user nova-compute[71317]:   </metadata>
Apr 14 04:34:44 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:34:44 user nova-compute[71317]:     <system>
Apr 14 04:34:44 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:34:44 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:34:44 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:34:44 user nova-compute[71317]:       <entry name="serial">970e81d2-c227-4eab-92dd-1ed732e67968</entry>
Apr 14 04:34:44 user nova-compute[71317]:       <entry name="uuid">970e81d2-c227-4eab-92dd-1ed732e67968</entry>
Apr 14 04:34:44 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:34:44 user nova-compute[71317]:     </system>
Apr 14 04:34:44 user nova-compute[71317]:   </sysinfo>
Apr 14 04:34:44 user nova-compute[71317]:   <os>
Apr 14 04:34:44 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:34:44 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:34:44 user nova-compute[71317]:   </os>
Apr 14 04:34:44 user nova-compute[71317]:   <features>
Apr 14 04:34:44 user nova-compute[71317]:     <acpi/>
Apr 14 04:34:44 user nova-compute[71317]:     <apic/>
Apr 14 04:34:44 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:34:44 user nova-compute[71317]:   </features>
Apr 14 04:34:44 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:34:44 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:34:44 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:34:44 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:34:44 user nova-compute[71317]:   </clock>
Apr 14 04:34:44 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:34:44 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:34:44 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:34:44 user nova-compute[71317]:   </cpu>
Apr 14 04:34:44 user nova-compute[71317]:   <devices>
Apr 14 04:34:44 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:34:44 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:34:44 user nova-compute[71317]:       <source dev="/dev/sdc"/>
Apr 14 04:34:44 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:34:44 user nova-compute[71317]:       <serial>9108333c-dbbf-4edb-a2b5-c6f034073970</serial>
Apr 14 04:34:44 user nova-compute[71317]:     </disk>
Apr 14 04:34:44 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:34:44 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:34:44 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.rescue"/>
Apr 14 04:34:44 user nova-compute[71317]:       <target dev="vdb" bus="virtio"/>
Apr 14 04:34:44 user nova-compute[71317]:       <boot order="1"/>
Apr 14 04:34:44 user nova-compute[71317]:     </disk>
Apr 14 04:34:44 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:34:44 user nova-compute[71317]:       <mac address="fa:16:3e:cd:43:7e"/>
Apr 14 04:34:44 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:34:44 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:34:44 user nova-compute[71317]:       <target dev="tap45a5a643-51"/>
Apr 14 04:34:44 user nova-compute[71317]:     </interface>
Apr 14 04:34:44 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:34:44 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/console.log" append="off"/>
Apr 14 04:34:44 user nova-compute[71317]:     </serial>
Apr 14 04:34:44 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:34:44 user nova-compute[71317]:     <video>
Apr 14 04:34:44 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:34:44 user nova-compute[71317]:     </video>
Apr 14 04:34:44 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:34:44 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:34:44 user nova-compute[71317]:     </rng>
Apr 14 04:34:44 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:34:44 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:34:44 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:34:44 user nova-compute[71317]:     </memballoon>
Apr 14 04:34:44 user nova-compute[71317]:   </devices>
Apr 14 04:34:44 user nova-compute[71317]: </domain>
Apr 14 04:34:44 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:34:44 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Instance destroyed successfully.
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] No VIF found with MAC fa:16:3e:cd:43:7e, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "283ef25b-314a-47c1-8414-9d7c737e5115" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:34:44 user nova-compute[71317]: INFO nova.compute.claims [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Claim successful on node user
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.compute.manager [req-8a2a9353-e785-4a46-9817-fd8fa1ac7951 req-7d614dd1-4438-473a-bc46-37c5df42a830 service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Received event network-vif-plugged-c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8a2a9353-e785-4a46-9817-fd8fa1ac7951 req-7d614dd1-4438-473a-bc46-37c5df42a830 service nova] Acquiring lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8a2a9353-e785-4a46-9817-fd8fa1ac7951 req-7d614dd1-4438-473a-bc46-37c5df42a830 service nova] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8a2a9353-e785-4a46-9817-fd8fa1ac7951 req-7d614dd1-4438-473a-bc46-37c5df42a830 service nova] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:44 user nova-compute[71317]: DEBUG nova.compute.manager [req-8a2a9353-e785-4a46-9817-fd8fa1ac7951 req-7d614dd1-4438-473a-bc46-37c5df42a830 service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] No waiting events found dispatching network-vif-plugged-c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:34:44 user nova-compute[71317]: WARNING nova.compute.manager [req-8a2a9353-e785-4a46-9817-fd8fa1ac7951 req-7d614dd1-4438-473a-bc46-37c5df42a830 service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Received unexpected event network-vif-plugged-c0df2242-82c7-408b-bdc2-ab608dee7c1a for instance with vm_state building and task_state spawning.
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446885.04113, 4b704133-1b1a-4579-857a-c74cd8b24fae => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:34:45 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] VM Resumed (Lifecycle Event)
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:34:45 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Instance spawned successfully.
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:34:45 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446885.0446784, 4b704133-1b1a-4579-857a-c74cd8b24fae => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:34:45 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] VM Started (Lifecycle Event)
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:34:45 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.457s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:34:45 user nova-compute[71317]: INFO nova.compute.manager [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Took 5.89 seconds to spawn the instance on the hypervisor.
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:34:45 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:34:45 user nova-compute[71317]: INFO nova.compute.manager [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Took 12.02 seconds to build instance.
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-070d1eeb-8692-4250-8cdd-485a0eca3d17 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 12.140s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.policy [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7cab2461ab0d4eb08b62feaf6396bc87', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '601e2806907a48d3995cc90b54da3c8c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:34:45 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Creating image(s)
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "/opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "/opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "/opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.055s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.162s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk 1073741824" returned: 0 in 0.047s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.216s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.166s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Checking if we can resize image /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:34:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-837d1c4c-9c70-48f3-b0a4-f0d2edc97687 req-dab68ab6-bbc1-49bd-a4d4-d09114376a62 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-837d1c4c-9c70-48f3-b0a4-f0d2edc97687 req-dab68ab6-bbc1-49bd-a4d4-d09114376a62 service nova] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-837d1c4c-9c70-48f3-b0a4-f0d2edc97687 req-dab68ab6-bbc1-49bd-a4d4-d09114376a62 service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-837d1c4c-9c70-48f3-b0a4-f0d2edc97687 req-dab68ab6-bbc1-49bd-a4d4-d09114376a62 service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG nova.compute.manager [req-837d1c4c-9c70-48f3-b0a4-f0d2edc97687 req-dab68ab6-bbc1-49bd-a4d4-d09114376a62 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] No waiting events found dispatching network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:34:46 user nova-compute[71317]: WARNING nova.compute.manager [req-837d1c4c-9c70-48f3-b0a4-f0d2edc97687 req-dab68ab6-bbc1-49bd-a4d4-d09114376a62 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received unexpected event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf for instance with vm_state active and task_state rescuing.
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Cannot resize image /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG nova.objects.instance [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'migration_context' on Instance uuid 283ef25b-314a-47c1-8414-9d7c737e5115 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Ensure instance console log exists: /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Successfully created port: 4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.rescue --force-share --output=json" returned: 0 in 0.145s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.rescue --force-share --output=json" returned: 0 in 0.145s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:34:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Successfully updated port: 4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "refresh_cache-283ef25b-314a-47c1-8414-9d7c737e5115" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquired lock "refresh_cache-283ef25b-314a-47c1-8414-9d7c737e5115" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-7815a330-bb9b-4c92-9b65-af9c66166957 req-eb7b0f75-0084-4087-8f72-d8ed59b7cb59 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received event network-changed-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.manager [req-7815a330-bb9b-4c92-9b65-af9c66166957 req-eb7b0f75-0084-4087-8f72-d8ed59b7cb59 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Refreshing instance network info cache due to event network-changed-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7815a330-bb9b-4c92-9b65-af9c66166957 req-eb7b0f75-0084-4087-8f72-d8ed59b7cb59 service nova] Acquiring lock "refresh_cache-283ef25b-314a-47c1-8414-9d7c737e5115" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:34:47 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:34:47 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8093MB free_disk=26.18450164794922GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 19a166de-f0c9-43f2-8c88-585e0d5d22cc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 390ca845-5660-49d0-bd8a-fcfc39176ca5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 970e81d2-c227-4eab-92dd-1ed732e67968 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 695d931b-1f30-4d57-b6d7-d199d2874e65 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3746b520-c10a-413e-9f1f-8807b3fd7c9a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 4b704133-1b1a-4579-857a-c74cd8b24fae actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 283ef25b-314a-47c1-8414-9d7c737e5115 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1792MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Updating instance_info_cache with network_info: [{"id": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "address": "fa:16:3e:6a:02:89", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4734e3dd-22", "ovs_interfaceid": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Releasing lock "refresh_cache-283ef25b-314a-47c1-8414-9d7c737e5115" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Instance network_info: |[{"id": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "address": "fa:16:3e:6a:02:89", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4734e3dd-22", "ovs_interfaceid": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7815a330-bb9b-4c92-9b65-af9c66166957 req-eb7b0f75-0084-4087-8f72-d8ed59b7cb59 service nova] Acquired lock "refresh_cache-283ef25b-314a-47c1-8414-9d7c737e5115" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.network.neutron [req-7815a330-bb9b-4c92-9b65-af9c66166957 req-eb7b0f75-0084-4087-8f72-d8ed59b7cb59 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Refreshing network info cache for port 4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Start _get_guest_xml network_info=[{"id": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "address": "fa:16:3e:6a:02:89", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4734e3dd-22", "ovs_interfaceid": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:34:47 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:34:47 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:34:47 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:34:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-930330581',display_name='tempest-AttachVolumeNegativeTest-server-930330581',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-930330581',id=39,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLkMrxFUn3dpVh/tPu0RpQlCyFBaJgCQyQYEwDhdWuIFHQhRjgZJy1F9LImzGW0YKeJb/lrhOq2b/KwQ6C50SgVO3YvlPJwAtkiYLlWF4eUkqOgYRYwPGHjor09Iuea8tQ==',key_name='tempest-keypair-499854002',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='601e2806907a48d3995cc90b54da3c8c',ramdisk_id='',reservation_id='r-ceukq3d3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1197484356',owner_user_name='tempest-AttachVolumeNegativeTest-1197484356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:34:45Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7cab2461ab0d4eb08b62feaf6396bc87',uuid=283ef25b-314a-47c1-8414-9d7c737e5115,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "address": "fa:16:3e:6a:02:89", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4734e3dd-22", "ovs_interfaceid": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converting VIF {"id": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "address": "fa:16:3e:6a:02:89", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4734e3dd-22", "ovs_interfaceid": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6a:02:89,bridge_name='br-int',has_traffic_filtering=True,id=4734e3dd-22a1-4d25-b2cd-e19ef4f66f83,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4734e3dd-22') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.objects.instance [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'pci_devices' on Instance uuid 283ef25b-314a-47c1-8414-9d7c737e5115 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:34:48 user nova-compute[71317]:   <uuid>283ef25b-314a-47c1-8414-9d7c737e5115</uuid>
Apr 14 04:34:48 user nova-compute[71317]:   <name>instance-00000027</name>
Apr 14 04:34:48 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:34:48 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:34:48 user nova-compute[71317]:   <metadata>
Apr 14 04:34:48 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:34:48 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:34:48 user nova-compute[71317]:       <nova:name>tempest-AttachVolumeNegativeTest-server-930330581</nova:name>
Apr 14 04:34:48 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:34:47</nova:creationTime>
Apr 14 04:34:48 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:34:48 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:34:48 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:34:48 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:34:48 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:34:48 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:34:48 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:34:48 user nova-compute[71317]:       <nova:owner>
Apr 14 04:34:48 user nova-compute[71317]:         <nova:user uuid="7cab2461ab0d4eb08b62feaf6396bc87">tempest-AttachVolumeNegativeTest-1197484356-project-member</nova:user>
Apr 14 04:34:48 user nova-compute[71317]:         <nova:project uuid="601e2806907a48d3995cc90b54da3c8c">tempest-AttachVolumeNegativeTest-1197484356</nova:project>
Apr 14 04:34:48 user nova-compute[71317]:       </nova:owner>
Apr 14 04:34:48 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:34:48 user nova-compute[71317]:       <nova:ports>
Apr 14 04:34:48 user nova-compute[71317]:         <nova:port uuid="4734e3dd-22a1-4d25-b2cd-e19ef4f66f83">
Apr 14 04:34:48 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 04:34:48 user nova-compute[71317]:         </nova:port>
Apr 14 04:34:48 user nova-compute[71317]:       </nova:ports>
Apr 14 04:34:48 user nova-compute[71317]:     </nova:instance>
Apr 14 04:34:48 user nova-compute[71317]:   </metadata>
Apr 14 04:34:48 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:34:48 user nova-compute[71317]:     <system>
Apr 14 04:34:48 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:34:48 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:34:48 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:34:48 user nova-compute[71317]:       <entry name="serial">283ef25b-314a-47c1-8414-9d7c737e5115</entry>
Apr 14 04:34:48 user nova-compute[71317]:       <entry name="uuid">283ef25b-314a-47c1-8414-9d7c737e5115</entry>
Apr 14 04:34:48 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:34:48 user nova-compute[71317]:     </system>
Apr 14 04:34:48 user nova-compute[71317]:   </sysinfo>
Apr 14 04:34:48 user nova-compute[71317]:   <os>
Apr 14 04:34:48 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:34:48 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:34:48 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:34:48 user nova-compute[71317]:   </os>
Apr 14 04:34:48 user nova-compute[71317]:   <features>
Apr 14 04:34:48 user nova-compute[71317]:     <acpi/>
Apr 14 04:34:48 user nova-compute[71317]:     <apic/>
Apr 14 04:34:48 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:34:48 user nova-compute[71317]:   </features>
Apr 14 04:34:48 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:34:48 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:34:48 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:34:48 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:34:48 user nova-compute[71317]:   </clock>
Apr 14 04:34:48 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:34:48 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:34:48 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:34:48 user nova-compute[71317]:   </cpu>
Apr 14 04:34:48 user nova-compute[71317]:   <devices>
Apr 14 04:34:48 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:34:48 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:34:48 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk"/>
Apr 14 04:34:48 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:34:48 user nova-compute[71317]:     </disk>
Apr 14 04:34:48 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:34:48 user nova-compute[71317]:       <mac address="fa:16:3e:6a:02:89"/>
Apr 14 04:34:48 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:34:48 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:34:48 user nova-compute[71317]:       <target dev="tap4734e3dd-22"/>
Apr 14 04:34:48 user nova-compute[71317]:     </interface>
Apr 14 04:34:48 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:34:48 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/console.log" append="off"/>
Apr 14 04:34:48 user nova-compute[71317]:     </serial>
Apr 14 04:34:48 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:34:48 user nova-compute[71317]:     <video>
Apr 14 04:34:48 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:34:48 user nova-compute[71317]:     </video>
Apr 14 04:34:48 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:34:48 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:34:48 user nova-compute[71317]:     </rng>
Apr 14 04:34:48 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:34:48 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:34:48 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:34:48 user nova-compute[71317]:     </memballoon>
Apr 14 04:34:48 user nova-compute[71317]:   </devices>
Apr 14 04:34:48 user nova-compute[71317]: </domain>
Apr 14 04:34:48 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:34:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-930330581',display_name='tempest-AttachVolumeNegativeTest-server-930330581',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-930330581',id=39,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLkMrxFUn3dpVh/tPu0RpQlCyFBaJgCQyQYEwDhdWuIFHQhRjgZJy1F9LImzGW0YKeJb/lrhOq2b/KwQ6C50SgVO3YvlPJwAtkiYLlWF4eUkqOgYRYwPGHjor09Iuea8tQ==',key_name='tempest-keypair-499854002',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='601e2806907a48d3995cc90b54da3c8c',ramdisk_id='',reservation_id='r-ceukq3d3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1197484356',owner_user_name='tempest-AttachVolumeNegativeTest-1197484356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:34:45Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7cab2461ab0d4eb08b62feaf6396bc87',uuid=283ef25b-314a-47c1-8414-9d7c737e5115,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "address": "fa:16:3e:6a:02:89", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4734e3dd-22", "ovs_interfaceid": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converting VIF {"id": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "address": "fa:16:3e:6a:02:89", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4734e3dd-22", "ovs_interfaceid": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6a:02:89,bridge_name='br-int',has_traffic_filtering=True,id=4734e3dd-22a1-4d25-b2cd-e19ef4f66f83,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4734e3dd-22') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG os_vif [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6a:02:89,bridge_name='br-int',has_traffic_filtering=True,id=4734e3dd-22a1-4d25-b2cd-e19ef4f66f83,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4734e3dd-22') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap4734e3dd-22, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap4734e3dd-22, col_values=(('external_ids', {'iface-id': '4734e3dd-22a1-4d25-b2cd-e19ef4f66f83', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6a:02:89', 'vm-uuid': '283ef25b-314a-47c1-8414-9d7c737e5115'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:48 user nova-compute[71317]: INFO os_vif [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6a:02:89,bridge_name='br-int',has_traffic_filtering=True,id=4734e3dd-22a1-4d25-b2cd-e19ef4f66f83,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4734e3dd-22')
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.617s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No VIF found with MAC fa:16:3e:6a:02:89, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-4d45c4bb-29b3-4ceb-b400-f01b352d44a7 req-a62554ae-dc49-4a99-b01c-66bbaefed0ab service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4d45c4bb-29b3-4ceb-b400-f01b352d44a7 req-a62554ae-dc49-4a99-b01c-66bbaefed0ab service nova] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4d45c4bb-29b3-4ceb-b400-f01b352d44a7 req-a62554ae-dc49-4a99-b01c-66bbaefed0ab service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4d45c4bb-29b3-4ceb-b400-f01b352d44a7 req-a62554ae-dc49-4a99-b01c-66bbaefed0ab service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.compute.manager [req-4d45c4bb-29b3-4ceb-b400-f01b352d44a7 req-a62554ae-dc49-4a99-b01c-66bbaefed0ab service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] No waiting events found dispatching network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:34:48 user nova-compute[71317]: WARNING nova.compute.manager [req-4d45c4bb-29b3-4ceb-b400-f01b352d44a7 req-a62554ae-dc49-4a99-b01c-66bbaefed0ab service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received unexpected event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf for instance with vm_state active and task_state rescuing.
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for 970e81d2-c227-4eab-92dd-1ed732e67968 due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446888.1767445, 970e81d2-c227-4eab-92dd-1ed732e67968 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:34:48 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] VM Resumed (Lifecycle Event)
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-80022fd8-44ab-4ce1-9229-bc5a24035e05 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:34:48 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446888.178386, 970e81d2-c227-4eab-92dd-1ed732e67968 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:34:48 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] VM Started (Lifecycle Event)
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.network.neutron [req-7815a330-bb9b-4c92-9b65-af9c66166957 req-eb7b0f75-0084-4087-8f72-d8ed59b7cb59 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Updated VIF entry in instance network info cache for port 4734e3dd-22a1-4d25-b2cd-e19ef4f66f83. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG nova.network.neutron [req-7815a330-bb9b-4c92-9b65-af9c66166957 req-eb7b0f75-0084-4087-8f72-d8ed59b7cb59 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Updating instance_info_cache with network_info: [{"id": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "address": "fa:16:3e:6a:02:89", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4734e3dd-22", "ovs_interfaceid": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7815a330-bb9b-4c92-9b65-af9c66166957 req-eb7b0f75-0084-4087-8f72-d8ed59b7cb59 service nova] Releasing lock "refresh_cache-283ef25b-314a-47c1-8414-9d7c737e5115" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:34:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:34:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:34:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:34:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:50 user nova-compute[71317]: DEBUG nova.compute.manager [req-b03a11aa-14ae-45e5-b082-353322938f86 req-2b5694ca-ab36-45fc-8580-31a0047e7e20 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received event network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:34:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b03a11aa-14ae-45e5-b082-353322938f86 req-2b5694ca-ab36-45fc-8580-31a0047e7e20 service nova] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b03a11aa-14ae-45e5-b082-353322938f86 req-2b5694ca-ab36-45fc-8580-31a0047e7e20 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b03a11aa-14ae-45e5-b082-353322938f86 req-2b5694ca-ab36-45fc-8580-31a0047e7e20 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:50 user nova-compute[71317]: DEBUG nova.compute.manager [req-b03a11aa-14ae-45e5-b082-353322938f86 req-2b5694ca-ab36-45fc-8580-31a0047e7e20 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] No waiting events found dispatching network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:34:50 user nova-compute[71317]: WARNING nova.compute.manager [req-b03a11aa-14ae-45e5-b082-353322938f86 req-2b5694ca-ab36-45fc-8580-31a0047e7e20 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received unexpected event network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 for instance with vm_state building and task_state spawning.
Apr 14 04:34:50 user nova-compute[71317]: DEBUG nova.compute.manager [req-b03a11aa-14ae-45e5-b082-353322938f86 req-2b5694ca-ab36-45fc-8580-31a0047e7e20 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received event network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:34:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b03a11aa-14ae-45e5-b082-353322938f86 req-2b5694ca-ab36-45fc-8580-31a0047e7e20 service nova] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b03a11aa-14ae-45e5-b082-353322938f86 req-2b5694ca-ab36-45fc-8580-31a0047e7e20 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b03a11aa-14ae-45e5-b082-353322938f86 req-2b5694ca-ab36-45fc-8580-31a0047e7e20 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:50 user nova-compute[71317]: DEBUG nova.compute.manager [req-b03a11aa-14ae-45e5-b082-353322938f86 req-2b5694ca-ab36-45fc-8580-31a0047e7e20 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] No waiting events found dispatching network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:34:50 user nova-compute[71317]: WARNING nova.compute.manager [req-b03a11aa-14ae-45e5-b082-353322938f86 req-2b5694ca-ab36-45fc-8580-31a0047e7e20 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received unexpected event network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 for instance with vm_state building and task_state spawning.
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446891.6970057, 283ef25b-314a-47c1-8414-9d7c737e5115 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:34:51 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] VM Resumed (Lifecycle Event)
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:34:51 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Instance spawned successfully.
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:34:51 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446891.7001998, 283ef25b-314a-47c1-8414-9d7c737e5115 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:34:51 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] VM Started (Lifecycle Event)
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:34:51 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:34:51 user nova-compute[71317]: INFO nova.compute.manager [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Took 6.44 seconds to spawn the instance on the hypervisor.
Apr 14 04:34:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:34:51 user nova-compute[71317]: INFO nova.compute.manager [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Took 7.21 seconds to build instance.
Apr 14 04:34:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1e47f64e-d6b5-4fa8-936f-b780a785eba4 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "283ef25b-314a-47c1-8414-9d7c737e5115" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.300s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-bf60a94d-38f0-4ecb-b1a6-afa0134392bd req-5ac7c3d7-5250-41bf-bcf1-f0727d637fd0 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Received event network-changed-bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:34:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-bf60a94d-38f0-4ecb-b1a6-afa0134392bd req-5ac7c3d7-5250-41bf-bcf1-f0727d637fd0 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Refreshing instance network info cache due to event network-changed-bc83f476-5176-4301-8272-80034f4a0daf. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:34:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bf60a94d-38f0-4ecb-b1a6-afa0134392bd req-5ac7c3d7-5250-41bf-bcf1-f0727d637fd0 service nova] Acquiring lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:34:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bf60a94d-38f0-4ecb-b1a6-afa0134392bd req-5ac7c3d7-5250-41bf-bcf1-f0727d637fd0 service nova] Acquired lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:34:57 user nova-compute[71317]: DEBUG nova.network.neutron [req-bf60a94d-38f0-4ecb-b1a6-afa0134392bd req-5ac7c3d7-5250-41bf-bcf1-f0727d637fd0 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Refreshing network info cache for port bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:34:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:58 user nova-compute[71317]: DEBUG nova.network.neutron [req-bf60a94d-38f0-4ecb-b1a6-afa0134392bd req-5ac7c3d7-5250-41bf-bcf1-f0727d637fd0 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Updated VIF entry in instance network info cache for port bc83f476-5176-4301-8272-80034f4a0daf. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:34:58 user nova-compute[71317]: DEBUG nova.network.neutron [req-bf60a94d-38f0-4ecb-b1a6-afa0134392bd req-5ac7c3d7-5250-41bf-bcf1-f0727d637fd0 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Updating instance_info_cache with network_info: [{"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:34:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bf60a94d-38f0-4ecb-b1a6-afa0134392bd req-5ac7c3d7-5250-41bf-bcf1-f0727d637fd0 service nova] Releasing lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:34:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG nova.compute.manager [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Stashing vm_state: active {{(pid=71317) _prep_resize /opt/stack/nova/nova/compute/manager.py:5499}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG nova.objects.instance [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'pci_requests' on Instance uuid 695d931b-1f30-4d57-b6d7-d199d2874e65 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:34:59 user nova-compute[71317]: INFO nova.compute.claims [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Claim successful on node user
Apr 14 04:34:59 user nova-compute[71317]: DEBUG nova.objects.instance [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'resources' on Instance uuid 695d931b-1f30-4d57-b6d7-d199d2874e65 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG nova.objects.instance [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'pci_devices' on Instance uuid 695d931b-1f30-4d57-b6d7-d199d2874e65 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:34:59 user nova-compute[71317]: INFO nova.compute.resource_tracker [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updating resource usage from migration 8795adac-f1fa-43a2-af3a-d19e669450ad
Apr 14 04:34:59 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: held 0.479s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:34:59 user nova-compute[71317]: INFO nova.compute.manager [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Migrating
Apr 14 04:34:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-c0f2f984-6874-4e41-954d-e20343ada0a3 req-7fc80f3d-2fbe-444e-986d-0c1860995921 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received event network-changed-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG nova.compute.manager [req-c0f2f984-6874-4e41-954d-e20343ada0a3 req-7fc80f3d-2fbe-444e-986d-0c1860995921 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Refreshing instance network info cache due to event network-changed-3562cd0d-8f80-4113-a397-b56e4455ef4a. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c0f2f984-6874-4e41-954d-e20343ada0a3 req-7fc80f3d-2fbe-444e-986d-0c1860995921 service nova] Acquiring lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c0f2f984-6874-4e41-954d-e20343ada0a3 req-7fc80f3d-2fbe-444e-986d-0c1860995921 service nova] Acquired lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG nova.network.neutron [req-c0f2f984-6874-4e41-954d-e20343ada0a3 req-7fc80f3d-2fbe-444e-986d-0c1860995921 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Refreshing network info cache for port 3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:34:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:35:00 user nova-compute[71317]: DEBUG nova.network.neutron [req-c0f2f984-6874-4e41-954d-e20343ada0a3 req-7fc80f3d-2fbe-444e-986d-0c1860995921 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updated VIF entry in instance network info cache for port 3562cd0d-8f80-4113-a397-b56e4455ef4a. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:35:00 user nova-compute[71317]: DEBUG nova.network.neutron [req-c0f2f984-6874-4e41-954d-e20343ada0a3 req-7fc80f3d-2fbe-444e-986d-0c1860995921 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updating instance_info_cache with network_info: [{"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:35:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c0f2f984-6874-4e41-954d-e20343ada0a3 req-7fc80f3d-2fbe-444e-986d-0c1860995921 service nova] Releasing lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:35:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquired lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:35:00 user nova-compute[71317]: DEBUG nova.network.neutron [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:35:01 user nova-compute[71317]: DEBUG nova.network.neutron [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updating instance_info_cache with network_info: [{"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:35:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Releasing lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:35:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Starting migrate_disk_and_power_off {{(pid=71317) migrate_disk_and_power_off /opt/stack/nova/nova/virt/libvirt/driver.py:11460}}
Apr 14 04:35:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd61a44b-dd49-44d9-b5d3-a9e3fdc909cd req-0c91bbfa-7cb7-42f9-ae6a-2b594805d5c8 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received event network-vif-unplugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:35:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd61a44b-dd49-44d9-b5d3-a9e3fdc909cd req-0c91bbfa-7cb7-42f9-ae6a-2b594805d5c8 service nova] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:35:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd61a44b-dd49-44d9-b5d3-a9e3fdc909cd req-0c91bbfa-7cb7-42f9-ae6a-2b594805d5c8 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:35:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-bd61a44b-dd49-44d9-b5d3-a9e3fdc909cd req-0c91bbfa-7cb7-42f9-ae6a-2b594805d5c8 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:35:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd61a44b-dd49-44d9-b5d3-a9e3fdc909cd req-0c91bbfa-7cb7-42f9-ae6a-2b594805d5c8 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] No waiting events found dispatching network-vif-unplugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:35:01 user nova-compute[71317]: WARNING nova.compute.manager [req-bd61a44b-dd49-44d9-b5d3-a9e3fdc909cd req-0c91bbfa-7cb7-42f9-ae6a-2b594805d5c8 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received unexpected event network-vif-unplugged-3562cd0d-8f80-4113-a397-b56e4455ef4a for instance with vm_state active and task_state resize_migrating.
Apr 14 04:35:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:02 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Instance destroyed successfully.
Apr 14 04:35:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:33:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-372327292',display_name='tempest-ServerActionsTestJSON-server-372327292',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-372327292',id=36,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:33:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-uxrcwktp',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='resize_migrating',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:34:59Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=695d931b-1f30-4d57-b6d7-d199d2874e65,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:f7:65:55"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:f7:65:55"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_vif [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3562cd0d-8f, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:02 user nova-compute[71317]: INFO os_vif [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f')
Apr 14 04:35:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sPgLhMjz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:35:02 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c48893d-9e5d-4b18-8a4a-01a9743ab46b]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:35:02 user nova-compute[71317]: No targets found.
Apr 14 04:35:02 user nova-compute[71317]: iSNS:
Apr 14 04:35:02 user nova-compute[71317]: No targets found.
Apr 14 04:35:02 user nova-compute[71317]: STATIC:
Apr 14 04:35:02 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269
Apr 14 04:35:02 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:35:02 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:35:02 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970
Apr 14 04:35:02 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:35:02 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:35:02 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75
Apr 14 04:35:02 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:35:02 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:35:02 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:35:02 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:35:02 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:35:02 user nova-compute[71317]: FIRMWARE:
Apr 14 04:35:02 user nova-compute[71317]: No targets found.
Apr 14 04:35:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:35:02 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:35:02 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[82f32bb0-8caa-442f-8d21-7d132c367d5c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[78298657-5a8d-48f5-a85d-656f0d35247f]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:35:02 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:35:02 user nova-compute[71317]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:35:02 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:35:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:35:02 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:35:02 user nova-compute[71317]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:35:02 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:35:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c41e5fc0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269'): ({'sdd'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Removing single pathed devices sdd {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2944f226-5584-448d-b8a5-66ce1e3deb46]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[84444e36-18a6-4798-ad59-d69278e1b7ef]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flushing IO for device /dev/sdd {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[88ea3fcd-c78f-4a6b-bb17-3284ae4285d8]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.040s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e2b841b-228d-4531-b3c1-a532a023942a]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-1SCST_BIOf616e73c-e1c3d039cc119196',) {{(pid=81937) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 14 04:35:02 user nova-compute[71317]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-1SCST_BIOf616e73c-e1c3d039cc119196
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[465c0c57-35cf-434a-a29c-54424e0ddad3]: (4, None) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c979c80-9a91-4ca2-8319-a3eb4f8faa3f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --logout" returned: 0 in 0.065s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9dfb87f2-83a6-4bac-bf5c-9620a9b7a639]: (4, ('Logging out of session [sid: 45, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260]\nLogout of [sid: 45, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 45, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260]
Apr 14 04:35:02 user nova-compute[71317]: Logout of [sid: 45, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260] successful.
Apr 14 04:35:02 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa1fca68-04b8-4b49-93aa-6a9707106431]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.317s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== disconnect_volume: return (318ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:35:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG nova.network.neutron [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Port 3562cd0d-8f80-4113-a397-b56e4455ef4a binding to destination host user is already ACTIVE {{(pid=71317) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3161}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquired lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG nova.network.neutron [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG nova.compute.manager [req-b02182a6-a89a-4f81-b596-64fe0482f66f req-354deccb-f9c4-48c8-9fad-806f04af73c0 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received event network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b02182a6-a89a-4f81-b596-64fe0482f66f req-354deccb-f9c4-48c8-9fad-806f04af73c0 service nova] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b02182a6-a89a-4f81-b596-64fe0482f66f req-354deccb-f9c4-48c8-9fad-806f04af73c0 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b02182a6-a89a-4f81-b596-64fe0482f66f req-354deccb-f9c4-48c8-9fad-806f04af73c0 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:35:03 user nova-compute[71317]: DEBUG nova.compute.manager [req-b02182a6-a89a-4f81-b596-64fe0482f66f req-354deccb-f9c4-48c8-9fad-806f04af73c0 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] No waiting events found dispatching network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:35:03 user nova-compute[71317]: WARNING nova.compute.manager [req-b02182a6-a89a-4f81-b596-64fe0482f66f req-354deccb-f9c4-48c8-9fad-806f04af73c0 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received unexpected event network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a for instance with vm_state active and task_state resize_migrated.
Apr 14 04:35:04 user nova-compute[71317]: DEBUG nova.network.neutron [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updating instance_info_cache with network_info: [{"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Releasing lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG os_brick.utils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[58a97d08-e983-4c61-b1bd-2295cb785e16]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[19172c91-f746-4a24-8452-ef7b4b771587]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[561069cd-a4c4-43a5-921a-235b370e7255]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8fcbaa62-3c40-4caf-8d06-2d81374dc9f7]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:35:04 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:35:04 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8188e15d-6b6f-4ad7-9d6e-40f5427c1feb]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:35:04 user nova-compute[71317]: DEBUG os_brick.utils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== get_connector_properties: return (124ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:35:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Starting finish_migration {{(pid=71317) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11647}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Instance directory exists: not creating {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 04:35:08 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Creating image(s)
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Ensure instance console log exists: /opt/stack/data/nova/instances/695d931b-1f30-4d57-b6d7-d199d2874e65/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Start _get_guest_xml network_info=[{"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:f7:65:55"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': True, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sPgLhMjz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': '695d931b-1f30-4d57-b6d7-d199d2874e65', 'attached_at': '2023-04-14T04:35:09.000000', 'detached_at': '', 'volume_id': '992a3f86-d313-4b1e-9491-3008be037269', 'serial': '992a3f86-d313-4b1e-9491-3008be037269'}, 'attachment_id': '330200a1-d9ad-42ed-be96-0bc76ac890b7', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:35:08 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:35:08 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:57Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='84',id=12,is_public=True,memory_mb=192,name='m1.micro',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.objects.instance [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'vcpu_model' on Instance uuid 695d931b-1f30-4d57-b6d7-d199d2874e65 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sPgLhMjz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:35:08 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:35:08 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:35:08 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2cb6c019-0cc1-4662-964d-dcf1141ae483]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:35:08 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a96ae03c-7b17-4c8a-916b-ab046c9a317a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269] added
Apr 14 04:35:08 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2b9870f-585a-4631-b007-78d5fb8b4286]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[adc91530-6659-4fd0-8b00-e92a42bb8ed7]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v sPgLhMjz {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v sPgLhMjz" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c92ffd9a-e397-4a0b-bf28-d664822722a9]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'sPgLhMjz'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1decad2-56bc-4852-8f87-ce826d8e106e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f9dcd3d-fa01-4a14-b2d3-8025c7fffbea]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:35:08 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:35:08 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:35:08 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:35:08 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:35:08 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:35:08 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:35:08 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[56e93781-cbf6-44ab-9049-15b0b3d8ab9f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260]
Apr 14 04:35:09 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260] successful.
Apr 14 04:35:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba65224f-80da-401b-b149-a71693739f41]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[760574c4-80c6-4a0b-b7a4-d1cd199c0404]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:35:09 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:35:09 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:35:09 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:35:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:35:09 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:35:09 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:35:09 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:35:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.276s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] HCTL ('35', '-', '-', 0) found on session 47 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[08b40173-b8a3-4a5e-898c-1410eab6b497]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Searching for a device in session 47 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:35:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sPgLhMjz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.310s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== connect_volume: return (1311ms) {'type': 'block', 'scsi_wwn': '26531633364303339', 'path': '/dev/sdd'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26531633364303339', 'path': '/dev/sdd'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:33:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-372327292',display_name='tempest-ServerActionsTestJSON-server-372327292',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-372327292',id=36,image_ref='',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:33:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-uxrcwktp',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:35:03Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=695d931b-1f30-4d57-b6d7-d199d2874e65,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:f7:65:55"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:f7:65:55"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:35:10 user nova-compute[71317]:   <uuid>695d931b-1f30-4d57-b6d7-d199d2874e65</uuid>
Apr 14 04:35:10 user nova-compute[71317]:   <name>instance-00000024</name>
Apr 14 04:35:10 user nova-compute[71317]:   <memory>196608</memory>
Apr 14 04:35:10 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:35:10 user nova-compute[71317]:   <metadata>
Apr 14 04:35:10 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:35:10 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:35:10 user nova-compute[71317]:       <nova:name>tempest-ServerActionsTestJSON-server-372327292</nova:name>
Apr 14 04:35:10 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:35:08</nova:creationTime>
Apr 14 04:35:10 user nova-compute[71317]:       <nova:flavor name="m1.micro">
Apr 14 04:35:10 user nova-compute[71317]:         <nova:memory>192</nova:memory>
Apr 14 04:35:10 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:35:10 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:35:10 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:35:10 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:35:10 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:35:10 user nova-compute[71317]:       <nova:owner>
Apr 14 04:35:10 user nova-compute[71317]:         <nova:user uuid="e35b3ae371cf4dd4b21e2999e58cd2f4">tempest-ServerActionsTestJSON-1048786641-project-member</nova:user>
Apr 14 04:35:10 user nova-compute[71317]:         <nova:project uuid="8c14c332dfcc44b49ddf3cb482dd0d4c">tempest-ServerActionsTestJSON-1048786641</nova:project>
Apr 14 04:35:10 user nova-compute[71317]:       </nova:owner>
Apr 14 04:35:10 user nova-compute[71317]:       <nova:ports>
Apr 14 04:35:10 user nova-compute[71317]:         <nova:port uuid="3562cd0d-8f80-4113-a397-b56e4455ef4a">
Apr 14 04:35:10 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 14 04:35:10 user nova-compute[71317]:         </nova:port>
Apr 14 04:35:10 user nova-compute[71317]:       </nova:ports>
Apr 14 04:35:10 user nova-compute[71317]:     </nova:instance>
Apr 14 04:35:10 user nova-compute[71317]:   </metadata>
Apr 14 04:35:10 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:35:10 user nova-compute[71317]:     <system>
Apr 14 04:35:10 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:35:10 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:35:10 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:35:10 user nova-compute[71317]:       <entry name="serial">695d931b-1f30-4d57-b6d7-d199d2874e65</entry>
Apr 14 04:35:10 user nova-compute[71317]:       <entry name="uuid">695d931b-1f30-4d57-b6d7-d199d2874e65</entry>
Apr 14 04:35:10 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:35:10 user nova-compute[71317]:     </system>
Apr 14 04:35:10 user nova-compute[71317]:   </sysinfo>
Apr 14 04:35:10 user nova-compute[71317]:   <os>
Apr 14 04:35:10 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:35:10 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:35:10 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:35:10 user nova-compute[71317]:   </os>
Apr 14 04:35:10 user nova-compute[71317]:   <features>
Apr 14 04:35:10 user nova-compute[71317]:     <acpi/>
Apr 14 04:35:10 user nova-compute[71317]:     <apic/>
Apr 14 04:35:10 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:35:10 user nova-compute[71317]:   </features>
Apr 14 04:35:10 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:35:10 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:35:10 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:35:10 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:35:10 user nova-compute[71317]:   </clock>
Apr 14 04:35:10 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:35:10 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:35:10 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:35:10 user nova-compute[71317]:   </cpu>
Apr 14 04:35:10 user nova-compute[71317]:   <devices>
Apr 14 04:35:10 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:35:10 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:35:10 user nova-compute[71317]:       <source dev="/dev/sdd"/>
Apr 14 04:35:10 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:35:10 user nova-compute[71317]:       <serial>992a3f86-d313-4b1e-9491-3008be037269</serial>
Apr 14 04:35:10 user nova-compute[71317]:     </disk>
Apr 14 04:35:10 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:35:10 user nova-compute[71317]:       <mac address="fa:16:3e:f7:65:55"/>
Apr 14 04:35:10 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:35:10 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:35:10 user nova-compute[71317]:       <target dev="tap3562cd0d-8f"/>
Apr 14 04:35:10 user nova-compute[71317]:     </interface>
Apr 14 04:35:10 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:35:10 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/695d931b-1f30-4d57-b6d7-d199d2874e65/console.log" append="off"/>
Apr 14 04:35:10 user nova-compute[71317]:     </serial>
Apr 14 04:35:10 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:35:10 user nova-compute[71317]:     <video>
Apr 14 04:35:10 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:35:10 user nova-compute[71317]:     </video>
Apr 14 04:35:10 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:35:10 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:35:10 user nova-compute[71317]:     </rng>
Apr 14 04:35:10 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:35:10 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:35:10 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:35:10 user nova-compute[71317]:     </memballoon>
Apr 14 04:35:10 user nova-compute[71317]:   </devices>
Apr 14 04:35:10 user nova-compute[71317]: </domain>
Apr 14 04:35:10 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:33:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-372327292',display_name='tempest-ServerActionsTestJSON-server-372327292',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-372327292',id=36,image_ref='',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:33:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-uxrcwktp',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:35:03Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=695d931b-1f30-4d57-b6d7-d199d2874e65,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:f7:65:55"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1431385934-network", "vif_mac": "fa:16:3e:f7:65:55"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG os_vif [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3562cd0d-8f, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap3562cd0d-8f, col_values=(('external_ids', {'iface-id': '3562cd0d-8f80-4113-a397-b56e4455ef4a', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f7:65:55', 'vm-uuid': '695d931b-1f30-4d57-b6d7-d199d2874e65'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:10 user nova-compute[71317]: INFO os_vif [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f')
Apr 14 04:35:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:35:10 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] No VIF found with MAC fa:16:3e:f7:65:55, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:35:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-7390266f-7c86-492a-8cff-6f8cec3044c6 req-0cc0eb98-ed53-4214-bfb8-71f9a6fc8da5 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received event network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:35:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7390266f-7c86-492a-8cff-6f8cec3044c6 req-0cc0eb98-ed53-4214-bfb8-71f9a6fc8da5 service nova] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:35:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7390266f-7c86-492a-8cff-6f8cec3044c6 req-0cc0eb98-ed53-4214-bfb8-71f9a6fc8da5 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:35:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7390266f-7c86-492a-8cff-6f8cec3044c6 req-0cc0eb98-ed53-4214-bfb8-71f9a6fc8da5 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:35:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-7390266f-7c86-492a-8cff-6f8cec3044c6 req-0cc0eb98-ed53-4214-bfb8-71f9a6fc8da5 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] No waiting events found dispatching network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:35:11 user nova-compute[71317]: WARNING nova.compute.manager [req-7390266f-7c86-492a-8cff-6f8cec3044c6 req-0cc0eb98-ed53-4214-bfb8-71f9a6fc8da5 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received unexpected event network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a for instance with vm_state active and task_state resize_finish.
Apr 14 04:35:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for 695d931b-1f30-4d57-b6d7-d199d2874e65 due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:35:13 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446913.5473597, 695d931b-1f30-4d57-b6d7-d199d2874e65 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:35:13 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] VM Resumed (Lifecycle Event)
Apr 14 04:35:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:35:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Instance running successfully.
Apr 14 04:35:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:35:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Failed to set time: agent not configured {{(pid=71317) sync_guest_time /opt/stack/nova/nova/virt/libvirt/guest.py:200}}
Apr 14 04:35:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-31933166-993d-4df1-ac8a-075e136e0307 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] finish_migration finished successfully. {{(pid=71317) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11742}}
Apr 14 04:35:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:35:13 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] During sync_power_state the instance has a pending task (resize_finish). Skip.
Apr 14 04:35:13 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446913.549554, 695d931b-1f30-4d57-b6d7-d199d2874e65 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:35:13 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] VM Started (Lifecycle Event)
Apr 14 04:35:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:35:13 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:35:13 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] During sync_power_state the instance has a pending task (resize_finish). Skip.
Apr 14 04:35:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5c209295-6d5a-45ec-9fa9-89fcbd130a06 req-285a6fec-edb6-4ee8-b3f8-79dd8185d140 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received event network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:35:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5c209295-6d5a-45ec-9fa9-89fcbd130a06 req-285a6fec-edb6-4ee8-b3f8-79dd8185d140 service nova] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:35:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5c209295-6d5a-45ec-9fa9-89fcbd130a06 req-285a6fec-edb6-4ee8-b3f8-79dd8185d140 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:35:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5c209295-6d5a-45ec-9fa9-89fcbd130a06 req-285a6fec-edb6-4ee8-b3f8-79dd8185d140 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:35:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-5c209295-6d5a-45ec-9fa9-89fcbd130a06 req-285a6fec-edb6-4ee8-b3f8-79dd8185d140 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] No waiting events found dispatching network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:35:13 user nova-compute[71317]: WARNING nova.compute.manager [req-5c209295-6d5a-45ec-9fa9-89fcbd130a06 req-285a6fec-edb6-4ee8-b3f8-79dd8185d140 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received unexpected event network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a for instance with vm_state active and task_state resize_finish.
Apr 14 04:35:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-8b16f0dc-d8eb-4f9e-b9bf-3c694b99b7e2 req-88dbbc2a-34fe-4090-a5c0-f71bedb8c675 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Received event network-changed-52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:35:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-8b16f0dc-d8eb-4f9e-b9bf-3c694b99b7e2 req-88dbbc2a-34fe-4090-a5c0-f71bedb8c675 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Refreshing instance network info cache due to event network-changed-52608de5-fde8-4fd9-81c0-10a060ab7130. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:35:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8b16f0dc-d8eb-4f9e-b9bf-3c694b99b7e2 req-88dbbc2a-34fe-4090-a5c0-f71bedb8c675 service nova] Acquiring lock "refresh_cache-3746b520-c10a-413e-9f1f-8807b3fd7c9a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:35:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8b16f0dc-d8eb-4f9e-b9bf-3c694b99b7e2 req-88dbbc2a-34fe-4090-a5c0-f71bedb8c675 service nova] Acquired lock "refresh_cache-3746b520-c10a-413e-9f1f-8807b3fd7c9a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:35:14 user nova-compute[71317]: DEBUG nova.network.neutron [req-8b16f0dc-d8eb-4f9e-b9bf-3c694b99b7e2 req-88dbbc2a-34fe-4090-a5c0-f71bedb8c675 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Refreshing network info cache for port 52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG nova.objects.instance [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lazy-loading 'flavor' on Instance uuid 3746b520-c10a-413e-9f1f-8807b3fd7c9a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.061s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG nova.network.neutron [req-8b16f0dc-d8eb-4f9e-b9bf-3c694b99b7e2 req-88dbbc2a-34fe-4090-a5c0-f71bedb8c675 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Updated VIF entry in instance network info cache for port 52608de5-fde8-4fd9-81c0-10a060ab7130. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG nova.network.neutron [req-8b16f0dc-d8eb-4f9e-b9bf-3c694b99b7e2 req-88dbbc2a-34fe-4090-a5c0-f71bedb8c675 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Updating instance_info_cache with network_info: [{"id": "52608de5-fde8-4fd9-81c0-10a060ab7130", "address": "fa:16:3e:36:5c:89", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.206", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52608de5-fd", "ovs_interfaceid": "52608de5-fde8-4fd9-81c0-10a060ab7130", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-8b16f0dc-d8eb-4f9e-b9bf-3c694b99b7e2 req-88dbbc2a-34fe-4090-a5c0-f71bedb8c675 service nova] Releasing lock "refresh_cache-3746b520-c10a-413e-9f1f-8807b3fd7c9a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:35:15 user nova-compute[71317]: INFO nova.compute.manager [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Attaching volume a61cb439-0424-4fe3-8c33-2b14401ebcc7 to /dev/vdb
Apr 14 04:35:15 user nova-compute[71317]: DEBUG os_brick.utils [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1ed0e77-b9a5-4151-aacb-3123d8b22afa]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a43f115-d736-49e2-bf0c-8ea2ef70df96]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7eda1b4f-3d09-4218-96e0-44070cd01b7c]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f60c4bf-781f-4408-8900-bbc743b08c72]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:35:15 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:35:15 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8afef3a7-3b9c-444d-ab9b-9c4cdb736fc0]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG os_brick.utils [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] <== get_connector_properties: return (121ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:35:15 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Updating existing volume attachment record: cea7ff72-8604-43c9-9b3e-27aea6d0a56b {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:35:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DFodRXjk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:35:20 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:35:20 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:35:20 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[918b3802-2747-49bd-ae4f-9c13f5541c5c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:35:20 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a9107aef-d935-4c6d-bd75-d72c91e616ff]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7] added
Apr 14 04:35:20 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3971438d-47c5-4dc9-9bd7-1273a9a99a29]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[aeb420cf-d504-415c-ab13-24ffe34d301e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v DFodRXjk {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v DFodRXjk" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[110f20d0-68ee-4537-b062-3ca9c17228d1]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'DFodRXjk'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[58d2a4a0-9e57-4286-ab63-71a68f0cb894]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e696c3e9-f96e-4ec6-ac81-e9af01560024]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --login" returned: 0 in 0.063s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec762aeb-a363-4565-a737-4a20ded07082]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7, portal: 172.16.0.220,3260]
Apr 14 04:35:20 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7, portal: 172.16.0.220,3260] successful.
Apr 14 04:35:20 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ca08140-bc4b-40ea-9430-45a421cdbef0]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[10c1a1ba-b296-459b-97c5-0e30b0b318b4]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 (non-flash)
Apr 14 04:35:20 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.289s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] HCTL ('37', '-', '-', 0) found on session 48 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb56ac07-9138-4e29-9483-4d43d8eac818]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:20 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Searching for a device in session 48 and hctl ['37', '*', '*', 0] yield: None {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:35:21 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Searching for a device in session 48 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:35:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DFodRXjk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:35:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.318s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:35:21 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] <== connect_volume: return (1320ms) {'type': 'block', 'scsi_wwn': '23164613839616266', 'path': '/dev/sdf'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:35:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23164613839616266', 'path': '/dev/sdf'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:35:21 user nova-compute[71317]: DEBUG nova.objects.instance [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lazy-loading 'flavor' on Instance uuid 3746b520-c10a-413e-9f1f-8807b3fd7c9a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:35:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:35:21 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:35:21 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:35:21 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:35:21 user nova-compute[71317]:   <serial>a61cb439-0424-4fe3-8c33-2b14401ebcc7</serial>
Apr 14 04:35:21 user nova-compute[71317]: </disk>
Apr 14 04:35:21 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:35:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:35:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:35:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] No VIF found with MAC fa:16:3e:36:5c:89, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:35:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-504be20c-4a25-424e-9d21-d34cf7d7fbf7 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.703s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:35:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:35:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:35:23 user nova-compute[71317]: INFO nova.compute.manager [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Detaching volume a61cb439-0424-4fe3-8c33-2b14401ebcc7
Apr 14 04:35:23 user nova-compute[71317]: INFO nova.virt.block_device [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Attempting to driver detach volume a61cb439-0424-4fe3-8c33-2b14401ebcc7 from mountpoint /dev/vdb
Apr 14 04:35:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Attempting to detach device vdb from instance 3746b520-c10a-413e-9f1f-8807b3fd7c9a from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:35:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:35:23 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:35:23 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:35:23 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:35:23 user nova-compute[71317]:   <serial>a61cb439-0424-4fe3-8c33-2b14401ebcc7</serial>
Apr 14 04:35:23 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:35:23 user nova-compute[71317]: </disk>
Apr 14 04:35:23 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:35:23 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Successfully detached device vdb from instance 3746b520-c10a-413e-9f1f-8807b3fd7c9a from the persistent domain config.
Apr 14 04:35:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 3746b520-c10a-413e-9f1f-8807b3fd7c9a from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:35:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:35:23 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:35:23 user nova-compute[71317]:   <source dev="/dev/sdf"/>
Apr 14 04:35:23 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:35:23 user nova-compute[71317]:   <serial>a61cb439-0424-4fe3-8c33-2b14401ebcc7</serial>
Apr 14 04:35:23 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:35:23 user nova-compute[71317]: </disk>
Apr 14 04:35:23 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681446924.1294713, 3746b520-c10a-413e-9f1f-8807b3fd7c9a => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 3746b520-c10a-413e-9f1f-8807b3fd7c9a {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:35:24 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Successfully detached device vdb from instance 3746b520-c10a-413e-9f1f-8807b3fd7c9a from the live domain config.
Apr 14 04:35:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DFodRXjk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:35:24 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1db9cff8-096f-40aa-894c-eb56d00005b2]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:35:24 user nova-compute[71317]: No targets found.
Apr 14 04:35:24 user nova-compute[71317]: iSNS:
Apr 14 04:35:24 user nova-compute[71317]: No targets found.
Apr 14 04:35:24 user nova-compute[71317]: STATIC:
Apr 14 04:35:24 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7
Apr 14 04:35:24 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:35:24 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:35:24 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269
Apr 14 04:35:24 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:35:24 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:35:24 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970
Apr 14 04:35:24 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:35:24 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:35:24 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75
Apr 14 04:35:24 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:35:24 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:35:24 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:35:24 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:35:24 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:35:24 user nova-compute[71317]: FIRMWARE:
Apr 14 04:35:24 user nova-compute[71317]: No targets found.
Apr 14 04:35:24 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:35:24 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:35:24 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6edcc017-948e-4cbd-abf9-d3e58986c00f]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[db534a74-67ef-45c6-84c8-aa5779e27162]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:35:24 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:35:24 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:35:24 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:35:24 user nova-compute[71317]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 (non-flash)
Apr 14 04:35:24 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:35:24 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:35:24 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:35:24 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:35:24 user nova-compute[71317]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 (non-flash)
Apr 14 04:35:24 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8490533d90>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7'): ({'sdf'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Removing single pathed devices sdf {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9b0c8ea-501e-4040-b41d-e12195443d8f]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7fd8d4e-2f23-4680-9a51-4ee46f8fd88e]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Flushing IO for device /dev/sdf {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[aaf9b5c9-b4c6-4995-ae13-6b9592cd2679]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.043s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7bf0177-74cc-4788-a86d-736e35f04a6c]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8cd2ad06-a37c-4902-a105-10d043ca6515]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b02eb70-d4da-4d74-9f53-10f8c3aaf0a7]: (4, ('Logging out of session [sid: 48, target: iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7, portal: 172.16.0.220,3260]\nLogout of [sid: 48, target: iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 48, target: iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7, portal: 172.16.0.220,3260]
Apr 14 04:35:24 user nova-compute[71317]: Logout of [sid: 48, target: iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7, portal: 172.16.0.220,3260] successful.
Apr 14 04:35:24 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a61cb439-0424-4fe3-8c33-2b14401ebcc7 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc63f33e-5788-4c8c-8200-fa4b22512e97]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.315s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] <== disconnect_volume: return (316ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:35:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:35:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:25 user nova-compute[71317]: DEBUG nova.objects.instance [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lazy-loading 'flavor' on Instance uuid 3746b520-c10a-413e-9f1f-8807b3fd7c9a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:35:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-224bc18a-8e01-444e-a640-74b0cee1e957 tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.662s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:35:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:32 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:35:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:35:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:36 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:35:36 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:35:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-19a166de-f0c9-43f2-8c88-585e0d5d22cc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:35:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-19a166de-f0c9-43f2-8c88-585e0d5d22cc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:35:36 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:35:36 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Updating instance_info_cache with network_info: [{"id": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "address": "fa:16:3e:b6:df:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape4576e1c-df", "ovs_interfaceid": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:35:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-19a166de-f0c9-43f2-8c88-585e0d5d22cc" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:35:36 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:35:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:39 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:35:40 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:35:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:41 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.rescue --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.rescue --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968/disk.rescue --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:35:45 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:35:45 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:35:45 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7774MB free_disk=26.09105682373047GB free_vcpus=2 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Applying migration context for instance 695d931b-1f30-4d57-b6d7-d199d2874e65 as it has an incoming, in-progress migration 8795adac-f1fa-43a2-af3a-d19e669450ad. Migration status is finished {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:950}}
Apr 14 04:35:45 user nova-compute[71317]: INFO nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updating resource usage from migration 8795adac-f1fa-43a2-af3a-d19e669450ad
Apr 14 04:35:45 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 19a166de-f0c9-43f2-8c88-585e0d5d22cc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 390ca845-5660-49d0-bd8a-fcfc39176ca5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:35:45 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:35:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 970e81d2-c227-4eab-92dd-1ed732e67968 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:35:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3746b520-c10a-413e-9f1f-8807b3fd7c9a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:35:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 4b704133-1b1a-4579-857a-c74cd8b24fae actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:35:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 283ef25b-314a-47c1-8414-9d7c737e5115 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:35:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Migration 8795adac-f1fa-43a2-af3a-d19e669450ad is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1637}}
Apr 14 04:35:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 695d931b-1f30-4d57-b6d7-d199d2874e65 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:35:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:35:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1984MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:35:46 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:35:46 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:35:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:35:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.492s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:35:47 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:35:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:35:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:35:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:35:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:35:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:35:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:35:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-7cc06f4d-8d8a-4ad4-9ffd-91ef87e4cd24 req-0fe0f0f2-3949-4fc9-87db-4851541e4b25 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Received event network-changed-52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-7cc06f4d-8d8a-4ad4-9ffd-91ef87e4cd24 req-0fe0f0f2-3949-4fc9-87db-4851541e4b25 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Refreshing instance network info cache due to event network-changed-52608de5-fde8-4fd9-81c0-10a060ab7130. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7cc06f4d-8d8a-4ad4-9ffd-91ef87e4cd24 req-0fe0f0f2-3949-4fc9-87db-4851541e4b25 service nova] Acquiring lock "refresh_cache-3746b520-c10a-413e-9f1f-8807b3fd7c9a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7cc06f4d-8d8a-4ad4-9ffd-91ef87e4cd24 req-0fe0f0f2-3949-4fc9-87db-4851541e4b25 service nova] Acquired lock "refresh_cache-3746b520-c10a-413e-9f1f-8807b3fd7c9a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG nova.network.neutron [req-7cc06f4d-8d8a-4ad4-9ffd-91ef87e4cd24 req-0fe0f0f2-3949-4fc9-87db-4851541e4b25 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Refreshing network info cache for port 52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:14 user nova-compute[71317]: INFO nova.compute.manager [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Terminating instance
Apr 14 04:36:14 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-cbc3eecd-bcc9-41f8-8b11-70152e842d71 req-9e0dc0bf-a06b-452c-be7f-6fe28e6c33f5 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Received event network-vif-unplugged-52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cbc3eecd-bcc9-41f8-8b11-70152e842d71 req-9e0dc0bf-a06b-452c-be7f-6fe28e6c33f5 service nova] Acquiring lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cbc3eecd-bcc9-41f8-8b11-70152e842d71 req-9e0dc0bf-a06b-452c-be7f-6fe28e6c33f5 service nova] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cbc3eecd-bcc9-41f8-8b11-70152e842d71 req-9e0dc0bf-a06b-452c-be7f-6fe28e6c33f5 service nova] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-cbc3eecd-bcc9-41f8-8b11-70152e842d71 req-9e0dc0bf-a06b-452c-be7f-6fe28e6c33f5 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] No waiting events found dispatching network-vif-unplugged-52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-cbc3eecd-bcc9-41f8-8b11-70152e842d71 req-9e0dc0bf-a06b-452c-be7f-6fe28e6c33f5 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Received event network-vif-unplugged-52608de5-fde8-4fd9-81c0-10a060ab7130 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG nova.network.neutron [req-7cc06f4d-8d8a-4ad4-9ffd-91ef87e4cd24 req-0fe0f0f2-3949-4fc9-87db-4851541e4b25 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Updated VIF entry in instance network info cache for port 52608de5-fde8-4fd9-81c0-10a060ab7130. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG nova.network.neutron [req-7cc06f4d-8d8a-4ad4-9ffd-91ef87e4cd24 req-0fe0f0f2-3949-4fc9-87db-4851541e4b25 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Updating instance_info_cache with network_info: [{"id": "52608de5-fde8-4fd9-81c0-10a060ab7130", "address": "fa:16:3e:36:5c:89", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52608de5-fd", "ovs_interfaceid": "52608de5-fde8-4fd9-81c0-10a060ab7130", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:36:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7cc06f4d-8d8a-4ad4-9ffd-91ef87e4cd24 req-0fe0f0f2-3949-4fc9-87db-4851541e4b25 service nova] Releasing lock "refresh_cache-3746b520-c10a-413e-9f1f-8807b3fd7c9a" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:15 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Instance destroyed successfully.
Apr 14 04:36:15 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lazy-loading 'resources' on Instance uuid 3746b520-c10a-413e-9f1f-8807b3fd7c9a {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:33:22Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestStampPattern-server-1660505126',display_name='tempest-TestStampPattern-server-1660505126',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-1660505126',id=37,image_ref='8666f9e6-ed56-46cf-a3bc-a12038eea805',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPy10nG4Vr3ssb6R59u94cqlTShzxPhKC2bNuw8xGLQYLxJf5JqesyHfE6AT1NUBBqZ5yv5gwPvV4khi3Sx9VWvIr6XsmH3ZmGby83ZqN/gypVHL9c9HqBsp1SNADUtbzA==',key_name='tempest-TestStampPattern-1877604753',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:33:29Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0e6175a4372245a4a57dca1f12f7f007',ramdisk_id='',reservation_id='r-w2epmlt6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_boot_roles='member,reader',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='390ca845-5660-49d0-bd8a-fcfc39176ca5',image_min_disk='1',image_min_ram='0',image_owner_id='0e6175a4372245a4a57dca1f12f7f007',image_owner_project_name='tempest-TestStampPattern-267011728',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-267011728-project-member',image_user_id='ebe3f41b95824cd78193d0bbe206a66b',owner_project_name='tempest-TestStampPattern-267011728',owner_user_name='tempest-TestStampPattern-267011728-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:33:30Z,user_data=None,user_id='ebe3f41b95824cd78193d0bbe206a66b',uuid=3746b520-c10a-413e-9f1f-8807b3fd7c9a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "52608de5-fde8-4fd9-81c0-10a060ab7130", "address": "fa:16:3e:36:5c:89", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.206", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52608de5-fd", "ovs_interfaceid": "52608de5-fde8-4fd9-81c0-10a060ab7130", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Converting VIF {"id": "52608de5-fde8-4fd9-81c0-10a060ab7130", "address": "fa:16:3e:36:5c:89", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.206", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52608de5-fd", "ovs_interfaceid": "52608de5-fde8-4fd9-81c0-10a060ab7130", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:36:5c:89,bridge_name='br-int',has_traffic_filtering=True,id=52608de5-fde8-4fd9-81c0-10a060ab7130,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52608de5-fd') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG os_vif [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:36:5c:89,bridge_name='br-int',has_traffic_filtering=True,id=52608de5-fde8-4fd9-81c0-10a060ab7130,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52608de5-fd') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap52608de5-fd, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:36:15 user nova-compute[71317]: INFO os_vif [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:36:5c:89,bridge_name='br-int',has_traffic_filtering=True,id=52608de5-fde8-4fd9-81c0-10a060ab7130,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52608de5-fd')
Apr 14 04:36:15 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Deleting instance files /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a_del
Apr 14 04:36:15 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Deletion of /opt/stack/data/nova/instances/3746b520-c10a-413e-9f1f-8807b3fd7c9a_del complete
Apr 14 04:36:15 user nova-compute[71317]: INFO nova.compute.manager [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Took 0.84 seconds to destroy the instance on the hypervisor.
Apr 14 04:36:15 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:36:15 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Took 0.45 seconds to deallocate network for instance.
Apr 14 04:36:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:16 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:36:16 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:36:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.421s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:16 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Deleted allocations for instance 3746b520-c10a-413e-9f1f-8807b3fd7c9a
Apr 14 04:36:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ac35b12c-39d0-4a1d-8a9a-7e70f60c267d tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.900s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:16 user nova-compute[71317]: DEBUG nova.compute.manager [req-0acfe2a2-54a2-4081-8d6a-80b99cd1298e req-6d4e1b9d-d980-48ce-a915-3a33ef147054 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Received event network-vif-deleted-52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:16 user nova-compute[71317]: DEBUG nova.compute.manager [req-92007df5-29fb-458c-9b0f-7db696dbfaa6 req-5a1d3cca-f943-471c-986c-5c89824cf9a3 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Received event network-vif-plugged-52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-92007df5-29fb-458c-9b0f-7db696dbfaa6 req-5a1d3cca-f943-471c-986c-5c89824cf9a3 service nova] Acquiring lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-92007df5-29fb-458c-9b0f-7db696dbfaa6 req-5a1d3cca-f943-471c-986c-5c89824cf9a3 service nova] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-92007df5-29fb-458c-9b0f-7db696dbfaa6 req-5a1d3cca-f943-471c-986c-5c89824cf9a3 service nova] Lock "3746b520-c10a-413e-9f1f-8807b3fd7c9a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:16 user nova-compute[71317]: DEBUG nova.compute.manager [req-92007df5-29fb-458c-9b0f-7db696dbfaa6 req-5a1d3cca-f943-471c-986c-5c89824cf9a3 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] No waiting events found dispatching network-vif-plugged-52608de5-fde8-4fd9-81c0-10a060ab7130 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:36:16 user nova-compute[71317]: WARNING nova.compute.manager [req-92007df5-29fb-458c-9b0f-7db696dbfaa6 req-5a1d3cca-f943-471c-986c-5c89824cf9a3 service nova] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Received unexpected event network-vif-plugged-52608de5-fde8-4fd9-81c0-10a060ab7130 for instance with vm_state deleted and task_state None.
Apr 14 04:36:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:36:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-0c1cc220-979d-426c-9079-e8503106bd45 req-82bc2afe-3158-454f-ac05-23e3635cb7fe service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Received event network-changed-c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-0c1cc220-979d-426c-9079-e8503106bd45 req-82bc2afe-3158-454f-ac05-23e3635cb7fe service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Refreshing instance network info cache due to event network-changed-c0df2242-82c7-408b-bdc2-ab608dee7c1a. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:36:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0c1cc220-979d-426c-9079-e8503106bd45 req-82bc2afe-3158-454f-ac05-23e3635cb7fe service nova] Acquiring lock "refresh_cache-4b704133-1b1a-4579-857a-c74cd8b24fae" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:36:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0c1cc220-979d-426c-9079-e8503106bd45 req-82bc2afe-3158-454f-ac05-23e3635cb7fe service nova] Acquired lock "refresh_cache-4b704133-1b1a-4579-857a-c74cd8b24fae" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:36:25 user nova-compute[71317]: DEBUG nova.network.neutron [req-0c1cc220-979d-426c-9079-e8503106bd45 req-82bc2afe-3158-454f-ac05-23e3635cb7fe service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Refreshing network info cache for port c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:36:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "4b704133-1b1a-4579-857a-c74cd8b24fae" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:25 user nova-compute[71317]: INFO nova.compute.manager [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Terminating instance
Apr 14 04:36:25 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG nova.network.neutron [req-0c1cc220-979d-426c-9079-e8503106bd45 req-82bc2afe-3158-454f-ac05-23e3635cb7fe service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Updated VIF entry in instance network info cache for port c0df2242-82c7-408b-bdc2-ab608dee7c1a. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG nova.network.neutron [req-0c1cc220-979d-426c-9079-e8503106bd45 req-82bc2afe-3158-454f-ac05-23e3635cb7fe service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Updating instance_info_cache with network_info: [{"id": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "address": "fa:16:3e:6d:f7:af", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.211", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0df2242-82", "ovs_interfaceid": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0c1cc220-979d-426c-9079-e8503106bd45 req-82bc2afe-3158-454f-ac05-23e3635cb7fe service nova] Releasing lock "refresh_cache-4b704133-1b1a-4579-857a-c74cd8b24fae" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-85050478-6397-496f-a826-dee14890c18b req-31da35c7-515b-4728-83eb-ceeed901764b service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Received event network-vif-unplugged-c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-85050478-6397-496f-a826-dee14890c18b req-31da35c7-515b-4728-83eb-ceeed901764b service nova] Acquiring lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-85050478-6397-496f-a826-dee14890c18b req-31da35c7-515b-4728-83eb-ceeed901764b service nova] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-85050478-6397-496f-a826-dee14890c18b req-31da35c7-515b-4728-83eb-ceeed901764b service nova] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-85050478-6397-496f-a826-dee14890c18b req-31da35c7-515b-4728-83eb-ceeed901764b service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] No waiting events found dispatching network-vif-unplugged-c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-85050478-6397-496f-a826-dee14890c18b req-31da35c7-515b-4728-83eb-ceeed901764b service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Received event network-vif-unplugged-c0df2242-82c7-408b-bdc2-ab608dee7c1a for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:36:26 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Instance destroyed successfully.
Apr 14 04:36:26 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lazy-loading 'resources' on Instance uuid 4b704133-1b1a-4579-857a-c74cd8b24fae {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:34:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1686619211',display_name='tempest-TestVolumeBootPattern-server-1686619211',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1686619211',id=38,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFZO4o4kQgpEvaFt8iKOk4kfW7YF9jZ6XS2jVS2UAmaxmR76rP/jf7a3pf4xOyQu5V8HIaLR//SNI2rVtnFikRInNPtRWTjIBvrk9d4sXMIgR6FP64tMb+FLBXcO2su56A==',key_name='tempest-TestVolumeBootPattern-1640497739',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:34:45Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-bxnjue41',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:34:45Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=4b704133-1b1a-4579-857a-c74cd8b24fae,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "address": "fa:16:3e:6d:f7:af", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0df2242-82", "ovs_interfaceid": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "address": "fa:16:3e:6d:f7:af", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0df2242-82", "ovs_interfaceid": "c0df2242-82c7-408b-bdc2-ab608dee7c1a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6d:f7:af,bridge_name='br-int',has_traffic_filtering=True,id=c0df2242-82c7-408b-bdc2-ab608dee7c1a,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0df2242-82') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_vif [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6d:f7:af,bridge_name='br-int',has_traffic_filtering=True,id=c0df2242-82c7-408b-bdc2-ab608dee7c1a,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0df2242-82') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc0df2242-82, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:36:26 user nova-compute[71317]: INFO os_vif [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6d:f7:af,bridge_name='br-int',has_traffic_filtering=True,id=c0df2242-82c7-408b-bdc2-ab608dee7c1a,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0df2242-82')
Apr 14 04:36:26 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tLxSpwaP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:36:26 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[673e8354-5684-4b8d-8930-0a5c9f720cec]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:36:26 user nova-compute[71317]: No targets found.
Apr 14 04:36:26 user nova-compute[71317]: iSNS:
Apr 14 04:36:26 user nova-compute[71317]: No targets found.
Apr 14 04:36:26 user nova-compute[71317]: STATIC:
Apr 14 04:36:26 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269
Apr 14 04:36:26 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:36:26 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:36:26 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970
Apr 14 04:36:26 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:36:26 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:36:26 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75
Apr 14 04:36:26 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:36:26 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:36:26 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:36:26 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:36:26 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:36:26 user nova-compute[71317]: FIRMWARE:
Apr 14 04:36:26 user nova-compute[71317]: No targets found.
Apr 14 04:36:26 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:36:26 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:36:26 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.094s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[affe3c77-b853-494c-91a4-d68bfb507937]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2db3cbc2-6500-4c3e-b6b1-12f04f0c1c7d]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:36:26 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:36:26 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:36:26 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:36:26 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:36:26 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:36:26 user nova-compute[71317]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:36:26 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:36:26 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84c43b4dc0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75'): ({'sde'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Removing single pathed devices sde {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad42591a-4aba-49b1-9cb8-95830bde884c]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0433e8d8-bc8a-4bd8-8384-00c4e96db448]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flushing IO for device /dev/sde {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd60edee-7b8e-4431-b8d5-ffca89b9de3c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[928fb80b-66be-4b80-926e-4ddd943ef9c8]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23132643462623363', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_a54339e2', '/dev/disk/by-id/scsi-1SCST_BIOa54339e2-12d4bb3c395f8b29') {{(pid=81937) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae89cb83-49c0-437e-b67e-5d9e0881e44f]: (4, None) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:36:26 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd67114c-875f-49b8-889d-2b87bb557001]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[861b32eb-056c-4cc5-a99c-efd97f22259c]: (4, ('Logging out of session [sid: 46, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260]\nLogout of [sid: 46, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 46, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260]
Apr 14 04:36:27 user nova-compute[71317]: Logout of [sid: 46, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260] successful.
Apr 14 04:36:27 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[56c06e57-45a1-4447-9d74-2e6979377c0d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.388s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== disconnect_volume: return (389ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:36:27 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Deleting instance files /opt/stack/data/nova/instances/4b704133-1b1a-4579-857a-c74cd8b24fae_del
Apr 14 04:36:27 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Deletion of /opt/stack/data/nova/instances/4b704133-1b1a-4579-857a-c74cd8b24fae_del complete
Apr 14 04:36:27 user nova-compute[71317]: INFO nova.compute.manager [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Took 1.23 seconds to destroy the instance on the hypervisor.
Apr 14 04:36:27 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:36:27 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:36:28 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Took 0.82 seconds to deallocate network for instance.
Apr 14 04:36:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-84a9a28e-9839-4d78-b06b-08673bc3c4ba req-0ce12f75-cae6-4bda-b704-cfa2cc8c9276 service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Received event network-vif-deleted-c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-4a76c0b6-d2e1-4cbb-8f0e-95cca4a3dab4 req-a8369cd6-8a41-4e1a-8048-74b7756013c4 service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Received event network-vif-plugged-c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4a76c0b6-d2e1-4cbb-8f0e-95cca4a3dab4 req-a8369cd6-8a41-4e1a-8048-74b7756013c4 service nova] Acquiring lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4a76c0b6-d2e1-4cbb-8f0e-95cca4a3dab4 req-a8369cd6-8a41-4e1a-8048-74b7756013c4 service nova] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4a76c0b6-d2e1-4cbb-8f0e-95cca4a3dab4 req-a8369cd6-8a41-4e1a-8048-74b7756013c4 service nova] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-4a76c0b6-d2e1-4cbb-8f0e-95cca4a3dab4 req-a8369cd6-8a41-4e1a-8048-74b7756013c4 service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] No waiting events found dispatching network-vif-plugged-c0df2242-82c7-408b-bdc2-ab608dee7c1a {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:36:28 user nova-compute[71317]: WARNING nova.compute.manager [req-4a76c0b6-d2e1-4cbb-8f0e-95cca4a3dab4 req-a8369cd6-8a41-4e1a-8048-74b7756013c4 service nova] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Received unexpected event network-vif-plugged-c0df2242-82c7-408b-bdc2-ab608dee7c1a for instance with vm_state active and task_state deleting.
Apr 14 04:36:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:28 user nova-compute[71317]: INFO nova.compute.manager [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Took 0.79 seconds to detach 1 volumes for instance.
Apr 14 04:36:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:29 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:36:29 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:36:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.307s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:29 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Deleted allocations for instance 4b704133-1b1a-4579-857a-c74cd8b24fae
Apr 14 04:36:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c59dd7d6-200a-460c-bd4f-3c0d2d1a3083 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "4b704133-1b1a-4579-857a-c74cd8b24fae" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.442s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:30 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446975.2827764, 3746b520-c10a-413e-9f1f-8807b3fd7c9a => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:36:30 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] VM Stopped (Lifecycle Event)
Apr 14 04:36:30 user nova-compute[71317]: DEBUG nova.compute.manager [None req-59901352-be50-47b6-ac17-30d9911329fc None None] [instance: 3746b520-c10a-413e-9f1f-8807b3fd7c9a] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:36:30 user nova-compute[71317]: INFO nova.compute.manager [None req-abb17ca3-6d6f-4570-916d-4de57970c9f6 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Unrescuing
Apr 14 04:36:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-abb17ca3-6d6f-4570-916d-4de57970c9f6 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "refresh_cache-970e81d2-c227-4eab-92dd-1ed732e67968" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:36:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-abb17ca3-6d6f-4570-916d-4de57970c9f6 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquired lock "refresh_cache-970e81d2-c227-4eab-92dd-1ed732e67968" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:36:30 user nova-compute[71317]: DEBUG nova.network.neutron [None req-abb17ca3-6d6f-4570-916d-4de57970c9f6 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:36:31 user nova-compute[71317]: DEBUG nova.network.neutron [None req-abb17ca3-6d6f-4570-916d-4de57970c9f6 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Updating instance_info_cache with network_info: [{"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:36:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-abb17ca3-6d6f-4570-916d-4de57970c9f6 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Releasing lock "refresh_cache-970e81d2-c227-4eab-92dd-1ed732e67968" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:36:31 user nova-compute[71317]: DEBUG nova.objects.instance [None req-abb17ca3-6d6f-4570-916d-4de57970c9f6 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'flavor' on Instance uuid 970e81d2-c227-4eab-92dd-1ed732e67968 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:36:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:31 user nova-compute[71317]: DEBUG nova.compute.manager [req-53d98b7c-e454-4fb9-8221-d8a60fe2f083 req-dea5cf08-7402-47a1-85f6-74152175da1e service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-unplugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-53d98b7c-e454-4fb9-8221-d8a60fe2f083 req-dea5cf08-7402-47a1-85f6-74152175da1e service nova] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-53d98b7c-e454-4fb9-8221-d8a60fe2f083 req-dea5cf08-7402-47a1-85f6-74152175da1e service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-53d98b7c-e454-4fb9-8221-d8a60fe2f083 req-dea5cf08-7402-47a1-85f6-74152175da1e service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:31 user nova-compute[71317]: DEBUG nova.compute.manager [req-53d98b7c-e454-4fb9-8221-d8a60fe2f083 req-dea5cf08-7402-47a1-85f6-74152175da1e service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] No waiting events found dispatching network-vif-unplugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:36:31 user nova-compute[71317]: WARNING nova.compute.manager [req-53d98b7c-e454-4fb9-8221-d8a60fe2f083 req-dea5cf08-7402-47a1-85f6-74152175da1e service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received unexpected event network-vif-unplugged-45a5a643-510a-45e2-a8e4-12eb53631daf for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:36:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:31 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Instance destroyed successfully.
Apr 14 04:36:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:33 user nova-compute[71317]: DEBUG nova.compute.manager [req-645bb7ad-8d1f-4f6b-b45e-d3c8968b748d req-c42ba268-bd35-4011-9081-f05c798e717d service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-645bb7ad-8d1f-4f6b-b45e-d3c8968b748d req-c42ba268-bd35-4011-9081-f05c798e717d service nova] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-645bb7ad-8d1f-4f6b-b45e-d3c8968b748d req-c42ba268-bd35-4011-9081-f05c798e717d service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-645bb7ad-8d1f-4f6b-b45e-d3c8968b748d req-c42ba268-bd35-4011-9081-f05c798e717d service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:33 user nova-compute[71317]: DEBUG nova.compute.manager [req-645bb7ad-8d1f-4f6b-b45e-d3c8968b748d req-c42ba268-bd35-4011-9081-f05c798e717d service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] No waiting events found dispatching network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:36:33 user nova-compute[71317]: WARNING nova.compute.manager [req-645bb7ad-8d1f-4f6b-b45e-d3c8968b748d req-c42ba268-bd35-4011-9081-f05c798e717d service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received unexpected event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:36:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:34 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-4a7baa64-4637-4048-8f26-a2cdecc1ae56 req-df24e36e-eed3-4539-926e-54524b3e335a service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4a7baa64-4637-4048-8f26-a2cdecc1ae56 req-df24e36e-eed3-4539-926e-54524b3e335a service nova] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4a7baa64-4637-4048-8f26-a2cdecc1ae56 req-df24e36e-eed3-4539-926e-54524b3e335a service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4a7baa64-4637-4048-8f26-a2cdecc1ae56 req-df24e36e-eed3-4539-926e-54524b3e335a service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-4a7baa64-4637-4048-8f26-a2cdecc1ae56 req-df24e36e-eed3-4539-926e-54524b3e335a service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] No waiting events found dispatching network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:36:35 user nova-compute[71317]: WARNING nova.compute.manager [req-4a7baa64-4637-4048-8f26-a2cdecc1ae56 req-df24e36e-eed3-4539-926e-54524b3e335a service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received unexpected event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-4a7baa64-4637-4048-8f26-a2cdecc1ae56 req-df24e36e-eed3-4539-926e-54524b3e335a service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4a7baa64-4637-4048-8f26-a2cdecc1ae56 req-df24e36e-eed3-4539-926e-54524b3e335a service nova] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4a7baa64-4637-4048-8f26-a2cdecc1ae56 req-df24e36e-eed3-4539-926e-54524b3e335a service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4a7baa64-4637-4048-8f26-a2cdecc1ae56 req-df24e36e-eed3-4539-926e-54524b3e335a service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-4a7baa64-4637-4048-8f26-a2cdecc1ae56 req-df24e36e-eed3-4539-926e-54524b3e335a service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] No waiting events found dispatching network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:36:35 user nova-compute[71317]: WARNING nova.compute.manager [req-4a7baa64-4637-4048-8f26-a2cdecc1ae56 req-df24e36e-eed3-4539-926e-54524b3e335a service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received unexpected event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf for instance with vm_state rescued and task_state unrescuing.
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-502a5a98-8a89-4639-9245-bdce28ee43cb req-34bbcf34-0a7f-4ce5-a9bb-6919fe09c8d5 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received event network-changed-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.compute.manager [req-502a5a98-8a89-4639-9245-bdce28ee43cb req-34bbcf34-0a7f-4ce5-a9bb-6919fe09c8d5 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Refreshing instance network info cache due to event network-changed-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-502a5a98-8a89-4639-9245-bdce28ee43cb req-34bbcf34-0a7f-4ce5-a9bb-6919fe09c8d5 service nova] Acquiring lock "refresh_cache-283ef25b-314a-47c1-8414-9d7c737e5115" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-502a5a98-8a89-4639-9245-bdce28ee43cb req-34bbcf34-0a7f-4ce5-a9bb-6919fe09c8d5 service nova] Acquired lock "refresh_cache-283ef25b-314a-47c1-8414-9d7c737e5115" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.network.neutron [req-502a5a98-8a89-4639-9245-bdce28ee43cb req-34bbcf34-0a7f-4ce5-a9bb-6919fe09c8d5 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Refreshing network info cache for port 4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.virt.libvirt.host [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Removed pending event for 970e81d2-c227-4eab-92dd-1ed732e67968 due to event {{(pid=71317) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446995.7389915, 970e81d2-c227-4eab-92dd-1ed732e67968 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:36:35 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] VM Resumed (Lifecycle Event)
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-abb17ca3-6d6f-4570-916d-4de57970c9f6 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:36:35 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681446995.7398217, 970e81d2-c227-4eab-92dd-1ed732e67968 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:36:35 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] VM Started (Lifecycle Event)
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:36:35 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:36:36 user nova-compute[71317]: DEBUG nova.network.neutron [req-502a5a98-8a89-4639-9245-bdce28ee43cb req-34bbcf34-0a7f-4ce5-a9bb-6919fe09c8d5 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Updated VIF entry in instance network info cache for port 4734e3dd-22a1-4d25-b2cd-e19ef4f66f83. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:36:36 user nova-compute[71317]: DEBUG nova.network.neutron [req-502a5a98-8a89-4639-9245-bdce28ee43cb req-34bbcf34-0a7f-4ce5-a9bb-6919fe09c8d5 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Updating instance_info_cache with network_info: [{"id": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "address": "fa:16:3e:6a:02:89", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.172", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4734e3dd-22", "ovs_interfaceid": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:36:36 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-502a5a98-8a89-4639-9245-bdce28ee43cb req-34bbcf34-0a7f-4ce5-a9bb-6919fe09c8d5 service nova] Releasing lock "refresh_cache-283ef25b-314a-47c1-8414-9d7c737e5115" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:36:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:37 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:36:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:36:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:36:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:36:37 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:36:37 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Updating instance_info_cache with network_info: [{"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.112", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:36:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:36:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "283ef25b-314a-47c1-8414-9d7c737e5115" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'flavor' on Instance uuid 283ef25b-314a-47c1-8414-9d7c737e5115 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "283ef25b-314a-47c1-8414-9d7c737e5115" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.049s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "283ef25b-314a-47c1-8414-9d7c737e5115" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:38 user nova-compute[71317]: INFO nova.compute.manager [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Attaching volume 5615b9ab-7102-4d44-a55e-0e2581c39757 to /dev/vdb
Apr 14 04:36:38 user nova-compute[71317]: DEBUG os_brick.utils [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[509d3c2d-0563-48bf-b3a8-c27c0d33d60b]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c252eb0-3219-4f44-9593-c1d673e2bc6f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ee1cae3-1ec9-4a19-82a1-a21cb4631100]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3ece29d-215e-4ed3-aae1-2a5d30087a22]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:36:38 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:36:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3355531-c870-4e1c-b069-ab8af3d458f6]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG os_brick.utils [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] <== get_connector_properties: return (125ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:36:38 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Updating existing volume attachment record: 9fdd36f5-c84d-4582-a392-992dcb0830ca {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:36:39 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:36:41 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681446986.6779966, 4b704133-1b1a-4579-857a-c74cd8b24fae => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:36:41 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] VM Stopped (Lifecycle Event)
Apr 14 04:36:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4bb21971-b4e9-43d7-b699-baa0be4a6b29 None None] [instance: 4b704133-1b1a-4579-857a-c74cd8b24fae] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:36:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:42 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TgIjxlbS', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:36:43 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:36:43 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:36:43 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8ea5673-236e-487f-9431-f1eb992d5b44]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:36:43 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[42982f7d-aafc-4ee0-9399-549357a775ba]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757] added
Apr 14 04:36:43 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb5999ea-018a-4fdd-80c3-2f6a73b6521f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[88123d03-8e5c-4b49-8cbd-90c4cb5773aa]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v TgIjxlbS {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v TgIjxlbS" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb0dd442-c7fb-47a4-9c35-550f5ce9592d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'TgIjxlbS'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad5f9be2-323a-44c7-af28-4191175b5487]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8128f782-85fa-49ce-b281-26241b8a0ccc]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --login" returned: 0 in 0.059s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5f6c17e-97b8-4018-bcf3-f035015f20ae]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757, portal: 172.16.0.220,3260]
Apr 14 04:36:43 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757, portal: 172.16.0.220,3260] successful.
Apr 14 04:36:43 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[50d76543-2be0-4cac-89b3-5e2ad2fffaa0]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[39d0c3f0-9b1f-4f7a-a453-c355ed3eaa32]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:36:43 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.270s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] HCTL ('36', '-', '-', 0) found on session 49 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[26e6f2d4-893b-4bc9-8880-6b391f7898ab]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Searching for a device in session 49 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TgIjxlbS', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:36:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.308s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] <== connect_volume: return (1311ms) {'type': 'block', 'scsi_wwn': '23133613436386163', 'path': '/dev/sde'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23133613436386163', 'path': '/dev/sde'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'flavor' on Instance uuid 283ef25b-314a-47c1-8414-9d7c737e5115 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:36:44 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:36:44 user nova-compute[71317]:   <source dev="/dev/sde"/>
Apr 14 04:36:44 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:36:44 user nova-compute[71317]:   <serial>5615b9ab-7102-4d44-a55e-0e2581c39757</serial>
Apr 14 04:36:44 user nova-compute[71317]: </disk>
Apr 14 04:36:44 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No VIF found with MAC fa:16:3e:6a:02:89, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2b8f4352-72ba-4870-8527-26f4acecd97c tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "283ef25b-314a-47c1-8414-9d7c737e5115" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.790s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:46 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:36:46 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8033MB free_disk=26.09003448486328GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Applying migration context for instance 695d931b-1f30-4d57-b6d7-d199d2874e65 as it has an incoming, in-progress migration 8795adac-f1fa-43a2-af3a-d19e669450ad. Migration status is finished {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:950}}
Apr 14 04:36:46 user nova-compute[71317]: INFO nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updating resource usage from migration 8795adac-f1fa-43a2-af3a-d19e669450ad
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 19a166de-f0c9-43f2-8c88-585e0d5d22cc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 390ca845-5660-49d0-bd8a-fcfc39176ca5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 970e81d2-c227-4eab-92dd-1ed732e67968 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 283ef25b-314a-47c1-8414-9d7c737e5115 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Migration 8795adac-f1fa-43a2-af3a-d19e669450ad is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1637}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 695d931b-1f30-4d57-b6d7-d199d2874e65 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1728MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:36:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.451s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:47 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:36:47 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:36:47 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:36:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:36:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65" by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65" acquired by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Going to confirm migration 2 {{(pid=71317) do_confirm_resize /opt/stack/nova/nova/compute/manager.py:4663}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquired lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG nova.objects.instance [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'info_cache' on Instance uuid 695d931b-1f30-4d57-b6d7-d199d2874e65 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:36:49 user nova-compute[71317]: INFO nova.compute.claims [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Claim successful on node user
Apr 14 04:36:49 user nova-compute[71317]: DEBUG nova.network.neutron [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updating instance_info_cache with network_info: [{"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Releasing lock "refresh_cache-695d931b-1f30-4d57-b6d7-d199d2874e65" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG nova.objects.instance [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'migration_context' on Instance uuid 695d931b-1f30-4d57-b6d7-d199d2874e65 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:36:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.452s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" :: waited 0.262s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG nova.network.neutron [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:36:50 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:36:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG nova.policy [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7cab2461ab0d4eb08b62feaf6396bc87', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '601e2806907a48d3995cc90b54da3c8c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:36:50 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Creating image(s)
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "/opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "/opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "/opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" :: held 0.465s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk 1073741824" returned: 0 in 0.048s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.200s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:50 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Deleted allocation for migration 8795adac-f1fa-43a2-af3a-d19e669450ad
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Checking if we can resize image /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:36:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-fbc7d81b-9e12-47bc-87a1-fe0139fcddb1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65" "released" by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" :: held 1.887s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:36:51 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Cannot resize image /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:36:51 user nova-compute[71317]: DEBUG nova.objects.instance [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'migration_context' on Instance uuid 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:36:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:36:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Ensure instance console log exists: /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:36:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.003s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:51 user nova-compute[71317]: DEBUG nova.network.neutron [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Successfully created port: 345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:36:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Successfully updated port: 345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "refresh_cache-6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquired lock "refresh_cache-6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-30b1be9f-83e1-4be2-970e-d996db7a9e34 req-1657be7d-1c1a-4e73-866a-8fda445b382c service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Received event network-changed-345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-30b1be9f-83e1-4be2-970e-d996db7a9e34 req-1657be7d-1c1a-4e73-866a-8fda445b382c service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Refreshing instance network info cache due to event network-changed-345c5341-edef-476a-9b94-e229dc1594b8. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-30b1be9f-83e1-4be2-970e-d996db7a9e34 req-1657be7d-1c1a-4e73-866a-8fda445b382c service nova] Acquiring lock "refresh_cache-6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Updating instance_info_cache with network_info: [{"id": "345c5341-edef-476a-9b94-e229dc1594b8", "address": "fa:16:3e:32:e2:76", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap345c5341-ed", "ovs_interfaceid": "345c5341-edef-476a-9b94-e229dc1594b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Releasing lock "refresh_cache-6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Instance network_info: |[{"id": "345c5341-edef-476a-9b94-e229dc1594b8", "address": "fa:16:3e:32:e2:76", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap345c5341-ed", "ovs_interfaceid": "345c5341-edef-476a-9b94-e229dc1594b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-30b1be9f-83e1-4be2-970e-d996db7a9e34 req-1657be7d-1c1a-4e73-866a-8fda445b382c service nova] Acquired lock "refresh_cache-6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.network.neutron [req-30b1be9f-83e1-4be2-970e-d996db7a9e34 req-1657be7d-1c1a-4e73-866a-8fda445b382c service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Refreshing network info cache for port 345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Start _get_guest_xml network_info=[{"id": "345c5341-edef-476a-9b94-e229dc1594b8", "address": "fa:16:3e:32:e2:76", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap345c5341-ed", "ovs_interfaceid": "345c5341-edef-476a-9b94-e229dc1594b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:36:52 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:36:52 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:36:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1738062211',display_name='tempest-AttachVolumeNegativeTest-server-1738062211',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1738062211',id=40,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAesg3BacL1BvdUPXS1xDD/3pqX31IW3Mm7TfS7lU8AJ9AncfmFmDbzZwUcbDyrA39s0koYymJjsrRF/BTnfmAyYfXAFKaGrYBH4j+YINr5ySxpEfDWQcwUiaidQx4WbYw==',key_name='tempest-keypair-2114462517',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='601e2806907a48d3995cc90b54da3c8c',ramdisk_id='',reservation_id='r-r5rv1iee',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1197484356',owner_user_name='tempest-AttachVolumeNegativeTest-1197484356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:36:50Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7cab2461ab0d4eb08b62feaf6396bc87',uuid=6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "345c5341-edef-476a-9b94-e229dc1594b8", "address": "fa:16:3e:32:e2:76", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap345c5341-ed", "ovs_interfaceid": "345c5341-edef-476a-9b94-e229dc1594b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converting VIF {"id": "345c5341-edef-476a-9b94-e229dc1594b8", "address": "fa:16:3e:32:e2:76", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap345c5341-ed", "ovs_interfaceid": "345c5341-edef-476a-9b94-e229dc1594b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:32:e2:76,bridge_name='br-int',has_traffic_filtering=True,id=345c5341-edef-476a-9b94-e229dc1594b8,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap345c5341-ed') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.objects.instance [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'pci_devices' on Instance uuid 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:36:52 user nova-compute[71317]:   <uuid>6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6</uuid>
Apr 14 04:36:52 user nova-compute[71317]:   <name>instance-00000028</name>
Apr 14 04:36:52 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:36:52 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:36:52 user nova-compute[71317]:   <metadata>
Apr 14 04:36:52 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:36:52 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:36:52 user nova-compute[71317]:       <nova:name>tempest-AttachVolumeNegativeTest-server-1738062211</nova:name>
Apr 14 04:36:52 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:36:52</nova:creationTime>
Apr 14 04:36:52 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:36:52 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:36:52 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:36:52 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:36:52 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:36:52 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:36:52 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:36:52 user nova-compute[71317]:       <nova:owner>
Apr 14 04:36:52 user nova-compute[71317]:         <nova:user uuid="7cab2461ab0d4eb08b62feaf6396bc87">tempest-AttachVolumeNegativeTest-1197484356-project-member</nova:user>
Apr 14 04:36:52 user nova-compute[71317]:         <nova:project uuid="601e2806907a48d3995cc90b54da3c8c">tempest-AttachVolumeNegativeTest-1197484356</nova:project>
Apr 14 04:36:52 user nova-compute[71317]:       </nova:owner>
Apr 14 04:36:52 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:36:52 user nova-compute[71317]:       <nova:ports>
Apr 14 04:36:52 user nova-compute[71317]:         <nova:port uuid="345c5341-edef-476a-9b94-e229dc1594b8">
Apr 14 04:36:52 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 04:36:52 user nova-compute[71317]:         </nova:port>
Apr 14 04:36:52 user nova-compute[71317]:       </nova:ports>
Apr 14 04:36:52 user nova-compute[71317]:     </nova:instance>
Apr 14 04:36:52 user nova-compute[71317]:   </metadata>
Apr 14 04:36:52 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:36:52 user nova-compute[71317]:     <system>
Apr 14 04:36:52 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:36:52 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:36:52 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:36:52 user nova-compute[71317]:       <entry name="serial">6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6</entry>
Apr 14 04:36:52 user nova-compute[71317]:       <entry name="uuid">6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6</entry>
Apr 14 04:36:52 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:36:52 user nova-compute[71317]:     </system>
Apr 14 04:36:52 user nova-compute[71317]:   </sysinfo>
Apr 14 04:36:52 user nova-compute[71317]:   <os>
Apr 14 04:36:52 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:36:52 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:36:52 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:36:52 user nova-compute[71317]:   </os>
Apr 14 04:36:52 user nova-compute[71317]:   <features>
Apr 14 04:36:52 user nova-compute[71317]:     <acpi/>
Apr 14 04:36:52 user nova-compute[71317]:     <apic/>
Apr 14 04:36:52 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:36:52 user nova-compute[71317]:   </features>
Apr 14 04:36:52 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:36:52 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:36:52 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:36:52 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:36:52 user nova-compute[71317]:   </clock>
Apr 14 04:36:52 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:36:52 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:36:52 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:36:52 user nova-compute[71317]:   </cpu>
Apr 14 04:36:52 user nova-compute[71317]:   <devices>
Apr 14 04:36:52 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:36:52 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:36:52 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk"/>
Apr 14 04:36:52 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:36:52 user nova-compute[71317]:     </disk>
Apr 14 04:36:52 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:36:52 user nova-compute[71317]:       <mac address="fa:16:3e:32:e2:76"/>
Apr 14 04:36:52 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:36:52 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:36:52 user nova-compute[71317]:       <target dev="tap345c5341-ed"/>
Apr 14 04:36:52 user nova-compute[71317]:     </interface>
Apr 14 04:36:52 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:36:52 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/console.log" append="off"/>
Apr 14 04:36:52 user nova-compute[71317]:     </serial>
Apr 14 04:36:52 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:36:52 user nova-compute[71317]:     <video>
Apr 14 04:36:52 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:36:52 user nova-compute[71317]:     </video>
Apr 14 04:36:52 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:36:52 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:36:52 user nova-compute[71317]:     </rng>
Apr 14 04:36:52 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:36:52 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:36:52 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:36:52 user nova-compute[71317]:     </memballoon>
Apr 14 04:36:52 user nova-compute[71317]:   </devices>
Apr 14 04:36:52 user nova-compute[71317]: </domain>
Apr 14 04:36:52 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:36:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1738062211',display_name='tempest-AttachVolumeNegativeTest-server-1738062211',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1738062211',id=40,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAesg3BacL1BvdUPXS1xDD/3pqX31IW3Mm7TfS7lU8AJ9AncfmFmDbzZwUcbDyrA39s0koYymJjsrRF/BTnfmAyYfXAFKaGrYBH4j+YINr5ySxpEfDWQcwUiaidQx4WbYw==',key_name='tempest-keypair-2114462517',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='601e2806907a48d3995cc90b54da3c8c',ramdisk_id='',reservation_id='r-r5rv1iee',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1197484356',owner_user_name='tempest-AttachVolumeNegativeTest-1197484356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:36:50Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7cab2461ab0d4eb08b62feaf6396bc87',uuid=6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "345c5341-edef-476a-9b94-e229dc1594b8", "address": "fa:16:3e:32:e2:76", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap345c5341-ed", "ovs_interfaceid": "345c5341-edef-476a-9b94-e229dc1594b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converting VIF {"id": "345c5341-edef-476a-9b94-e229dc1594b8", "address": "fa:16:3e:32:e2:76", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap345c5341-ed", "ovs_interfaceid": "345c5341-edef-476a-9b94-e229dc1594b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:32:e2:76,bridge_name='br-int',has_traffic_filtering=True,id=345c5341-edef-476a-9b94-e229dc1594b8,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap345c5341-ed') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG os_vif [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:32:e2:76,bridge_name='br-int',has_traffic_filtering=True,id=345c5341-edef-476a-9b94-e229dc1594b8,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap345c5341-ed') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap345c5341-ed, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap345c5341-ed, col_values=(('external_ids', {'iface-id': '345c5341-edef-476a-9b94-e229dc1594b8', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:32:e2:76', 'vm-uuid': '6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:52 user nova-compute[71317]: INFO os_vif [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:32:e2:76,bridge_name='br-int',has_traffic_filtering=True,id=345c5341-edef-476a-9b94-e229dc1594b8,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap345c5341-ed')
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:36:52 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No VIF found with MAC fa:16:3e:32:e2:76, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:36:53 user nova-compute[71317]: DEBUG nova.network.neutron [req-30b1be9f-83e1-4be2-970e-d996db7a9e34 req-1657be7d-1c1a-4e73-866a-8fda445b382c service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Updated VIF entry in instance network info cache for port 345c5341-edef-476a-9b94-e229dc1594b8. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:36:53 user nova-compute[71317]: DEBUG nova.network.neutron [req-30b1be9f-83e1-4be2-970e-d996db7a9e34 req-1657be7d-1c1a-4e73-866a-8fda445b382c service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Updating instance_info_cache with network_info: [{"id": "345c5341-edef-476a-9b94-e229dc1594b8", "address": "fa:16:3e:32:e2:76", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap345c5341-ed", "ovs_interfaceid": "345c5341-edef-476a-9b94-e229dc1594b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:36:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-30b1be9f-83e1-4be2-970e-d996db7a9e34 req-1657be7d-1c1a-4e73-866a-8fda445b382c service nova] Releasing lock "refresh_cache-6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:36:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-a5ba5678-0fd8-411c-b424-909df7426f48 req-56b072ce-d2ff-4c34-ae8a-2d2cc1ba60cc service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Received event network-vif-plugged-345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a5ba5678-0fd8-411c-b424-909df7426f48 req-56b072ce-d2ff-4c34-ae8a-2d2cc1ba60cc service nova] Acquiring lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a5ba5678-0fd8-411c-b424-909df7426f48 req-56b072ce-d2ff-4c34-ae8a-2d2cc1ba60cc service nova] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a5ba5678-0fd8-411c-b424-909df7426f48 req-56b072ce-d2ff-4c34-ae8a-2d2cc1ba60cc service nova] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-a5ba5678-0fd8-411c-b424-909df7426f48 req-56b072ce-d2ff-4c34-ae8a-2d2cc1ba60cc service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] No waiting events found dispatching network-vif-plugged-345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:36:54 user nova-compute[71317]: WARNING nova.compute.manager [req-a5ba5678-0fd8-411c-b424-909df7426f48 req-56b072ce-d2ff-4c34-ae8a-2d2cc1ba60cc service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Received unexpected event network-vif-plugged-345c5341-edef-476a-9b94-e229dc1594b8 for instance with vm_state building and task_state spawning.
Apr 14 04:36:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447016.439463, 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:36:56 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] VM Resumed (Lifecycle Event)
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:36:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Instance spawned successfully.
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:36:56 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447016.4408333, 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:36:56 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] VM Started (Lifecycle Event)
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-f0fa6887-3e93-452a-9b4e-a5271b32fcf5 req-8df70c30-1014-443f-8700-596790fdeab2 service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Received event network-vif-plugged-345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0fa6887-3e93-452a-9b4e-a5271b32fcf5 req-8df70c30-1014-443f-8700-596790fdeab2 service nova] Acquiring lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0fa6887-3e93-452a-9b4e-a5271b32fcf5 req-8df70c30-1014-443f-8700-596790fdeab2 service nova] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0fa6887-3e93-452a-9b4e-a5271b32fcf5 req-8df70c30-1014-443f-8700-596790fdeab2 service nova] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-f0fa6887-3e93-452a-9b4e-a5271b32fcf5 req-8df70c30-1014-443f-8700-596790fdeab2 service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] No waiting events found dispatching network-vif-plugged-345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:36:56 user nova-compute[71317]: WARNING nova.compute.manager [req-f0fa6887-3e93-452a-9b4e-a5271b32fcf5 req-8df70c30-1014-443f-8700-596790fdeab2 service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Received unexpected event network-vif-plugged-345c5341-edef-476a-9b94-e229dc1594b8 for instance with vm_state building and task_state spawning.
Apr 14 04:36:56 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:36:56 user nova-compute[71317]: INFO nova.compute.manager [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Took 6.20 seconds to spawn the instance on the hypervisor.
Apr 14 04:36:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:36:56 user nova-compute[71317]: INFO nova.compute.manager [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Took 7.01 seconds to build instance.
Apr 14 04:36:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-39cb43c6-9bf2-464c-b19f-27a820000f27 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.112s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:36:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:36:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-39913795-598c-457c-a229-bdcb0353106f req-e4bf2211-bb62-4371-8411-09f540f9de43 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Received event network-changed-488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-39913795-598c-457c-a229-bdcb0353106f req-e4bf2211-bb62-4371-8411-09f540f9de43 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Refreshing instance network info cache due to event network-changed-488d7017-4718-4554-80f0-7fb119efc417. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-39913795-598c-457c-a229-bdcb0353106f req-e4bf2211-bb62-4371-8411-09f540f9de43 service nova] Acquiring lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-39913795-598c-457c-a229-bdcb0353106f req-e4bf2211-bb62-4371-8411-09f540f9de43 service nova] Acquired lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG nova.network.neutron [req-39913795-598c-457c-a229-bdcb0353106f req-e4bf2211-bb62-4371-8411-09f540f9de43 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Refreshing network info cache for port 488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:37:12 user nova-compute[71317]: INFO nova.compute.manager [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Terminating instance
Apr 14 04:37:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-815d7c49-ddf3-487a-b3a5-1055791b7596 req-7162f005-2f0b-45df-8946-eefbf5d3405d service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Received event network-vif-unplugged-488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-815d7c49-ddf3-487a-b3a5-1055791b7596 req-7162f005-2f0b-45df-8946-eefbf5d3405d service nova] Acquiring lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-815d7c49-ddf3-487a-b3a5-1055791b7596 req-7162f005-2f0b-45df-8946-eefbf5d3405d service nova] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-815d7c49-ddf3-487a-b3a5-1055791b7596 req-7162f005-2f0b-45df-8946-eefbf5d3405d service nova] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-815d7c49-ddf3-487a-b3a5-1055791b7596 req-7162f005-2f0b-45df-8946-eefbf5d3405d service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] No waiting events found dispatching network-vif-unplugged-488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-815d7c49-ddf3-487a-b3a5-1055791b7596 req-7162f005-2f0b-45df-8946-eefbf5d3405d service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Received event network-vif-unplugged-488d7017-4718-4554-80f0-7fb119efc417 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG nova.network.neutron [req-39913795-598c-457c-a229-bdcb0353106f req-e4bf2211-bb62-4371-8411-09f540f9de43 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Updated VIF entry in instance network info cache for port 488d7017-4718-4554-80f0-7fb119efc417. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG nova.network.neutron [req-39913795-598c-457c-a229-bdcb0353106f req-e4bf2211-bb62-4371-8411-09f540f9de43 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Updating instance_info_cache with network_info: [{"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-39913795-598c-457c-a229-bdcb0353106f req-e4bf2211-bb62-4371-8411-09f540f9de43 service nova] Releasing lock "refresh_cache-390ca845-5660-49d0-bd8a-fcfc39176ca5" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:37:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Instance destroyed successfully.
Apr 14 04:37:13 user nova-compute[71317]: DEBUG nova.objects.instance [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lazy-loading 'resources' on Instance uuid 390ca845-5660-49d0-bd8a-fcfc39176ca5 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:37:13 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:30:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestStampPattern-server-530656870',display_name='tempest-TestStampPattern-server-530656870',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-530656870',id=32,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPy10nG4Vr3ssb6R59u94cqlTShzxPhKC2bNuw8xGLQYLxJf5JqesyHfE6AT1NUBBqZ5yv5gwPvV4khi3Sx9VWvIr6XsmH3ZmGby83ZqN/gypVHL9c9HqBsp1SNADUtbzA==',key_name='tempest-TestStampPattern-1877604753',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:30:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0e6175a4372245a4a57dca1f12f7f007',ramdisk_id='',reservation_id='r-xhxn1384',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestStampPattern-267011728',owner_user_name='tempest-TestStampPattern-267011728-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:33:17Z,user_data=None,user_id='ebe3f41b95824cd78193d0bbe206a66b',uuid=390ca845-5660-49d0-bd8a-fcfc39176ca5,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.112", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:37:13 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Converting VIF {"id": "488d7017-4718-4554-80f0-7fb119efc417", "address": "fa:16:3e:2f:d9:94", "network": {"id": "1ba7d569-454d-4c89-80be-245f8ed0b1f9", "bridge": "br-int", "label": "tempest-TestStampPattern-1837448162-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.112", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0e6175a4372245a4a57dca1f12f7f007", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap488d7017-47", "ovs_interfaceid": "488d7017-4718-4554-80f0-7fb119efc417", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:37:13 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:2f:d9:94,bridge_name='br-int',has_traffic_filtering=True,id=488d7017-4718-4554-80f0-7fb119efc417,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap488d7017-47') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:37:13 user nova-compute[71317]: DEBUG os_vif [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:2f:d9:94,bridge_name='br-int',has_traffic_filtering=True,id=488d7017-4718-4554-80f0-7fb119efc417,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap488d7017-47') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:37:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap488d7017-47, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:37:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:37:13 user nova-compute[71317]: INFO os_vif [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:2f:d9:94,bridge_name='br-int',has_traffic_filtering=True,id=488d7017-4718-4554-80f0-7fb119efc417,network=Network(1ba7d569-454d-4c89-80be-245f8ed0b1f9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap488d7017-47')
Apr 14 04:37:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Deleting instance files /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5_del
Apr 14 04:37:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Deletion of /opt/stack/data/nova/instances/390ca845-5660-49d0-bd8a-fcfc39176ca5_del complete
Apr 14 04:37:13 user nova-compute[71317]: INFO nova.compute.manager [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Took 0.86 seconds to destroy the instance on the hypervisor.
Apr 14 04:37:13 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:37:13 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:37:13 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:37:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:14 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:37:14 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Took 0.76 seconds to deallocate network for instance.
Apr 14 04:37:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:37:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:37:14 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:37:14 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:37:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.310s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:37:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-0f41d822-68ea-41bf-a759-8bcef1666e7e req-a9e082c8-9299-4f1b-a3e6-03f9bedad223 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Received event network-vif-deleted-488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:37:14 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Deleted allocations for instance 390ca845-5660-49d0-bd8a-fcfc39176ca5
Apr 14 04:37:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-bd57c173-ba39-4398-ac92-c1a2d9c3790a tempest-TestStampPattern-267011728 tempest-TestStampPattern-267011728-project-member] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.113s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:37:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f1b11b7-92a3-42c4-9f37-e5185c83e7c5 req-c6a1248c-cdd5-4738-b65d-f153c6e1f246 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Received event network-vif-plugged-488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:37:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f1b11b7-92a3-42c4-9f37-e5185c83e7c5 req-c6a1248c-cdd5-4738-b65d-f153c6e1f246 service nova] Acquiring lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:37:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f1b11b7-92a3-42c4-9f37-e5185c83e7c5 req-c6a1248c-cdd5-4738-b65d-f153c6e1f246 service nova] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:37:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5f1b11b7-92a3-42c4-9f37-e5185c83e7c5 req-c6a1248c-cdd5-4738-b65d-f153c6e1f246 service nova] Lock "390ca845-5660-49d0-bd8a-fcfc39176ca5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:37:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-5f1b11b7-92a3-42c4-9f37-e5185c83e7c5 req-c6a1248c-cdd5-4738-b65d-f153c6e1f246 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] No waiting events found dispatching network-vif-plugged-488d7017-4718-4554-80f0-7fb119efc417 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:37:14 user nova-compute[71317]: WARNING nova.compute.manager [req-5f1b11b7-92a3-42c4-9f37-e5185c83e7c5 req-c6a1248c-cdd5-4738-b65d-f153c6e1f246 service nova] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Received unexpected event network-vif-plugged-488d7017-4718-4554-80f0-7fb119efc417 for instance with vm_state deleted and task_state None.
Apr 14 04:37:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:37:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:37:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:37:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:37:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:37:17 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:37:17 user nova-compute[71317]: INFO nova.compute.claims [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Claim successful on node user
Apr 14 04:37:17 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:37:17 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.358s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:37:18 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:37:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:37:18 user nova-compute[71317]: INFO nova.virt.block_device [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Booting with volume 495f173c-6873-42b6-a0fe-42da67f3aa75 at /dev/vda
Apr 14 04:37:18 user nova-compute[71317]: DEBUG nova.policy [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bcf79de7da27495dbfd14c556b5b54c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '693d95858c624feaa60114aa89082186', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG os_brick.utils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[524bfba2-666a-42c7-810b-4d46c2a9ccd2]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a4b4f9c-23c8-40a5-a652-52fa5a85a896]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[403b9191-9f58-45d5-8b04-c5ba01abd97f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[037ba06f-ab2e-44aa-8d1d-32a273eaa0b0]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:37:18 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:37:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9377700f-f50f-4fb8-833b-53a37205b2d3]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG os_brick.utils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== get_connector_properties: return (118ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updating existing volume attachment record: 5a96efd1-e24e-41bb-aed3-6469de3a07a1 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:19 user nova-compute[71317]: DEBUG nova.network.neutron [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Successfully created port: f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:37:19 user nova-compute[71317]: DEBUG nova.network.neutron [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Successfully updated port: f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:37:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:37:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquired lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:37:19 user nova-compute[71317]: DEBUG nova.network.neutron [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:37:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-7e970dc1-d44c-4f78-98e9-10bed5a258b7 req-97a97174-4dc6-4cad-9fc7-fd3321cde257 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Received event network-changed-f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:37:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-7e970dc1-d44c-4f78-98e9-10bed5a258b7 req-97a97174-4dc6-4cad-9fc7-fd3321cde257 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Refreshing instance network info cache due to event network-changed-f204cf26-ba59-48a1-b08e-6d8915075961. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:37:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7e970dc1-d44c-4f78-98e9-10bed5a258b7 req-97a97174-4dc6-4cad-9fc7-fd3321cde257 service nova] Acquiring lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:37:19 user nova-compute[71317]: DEBUG nova.network.neutron [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:37:20 user nova-compute[71317]: DEBUG nova.network.neutron [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updating instance_info_cache with network_info: [{"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:37:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Releasing lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:37:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Instance network_info: |[{"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:37:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7e970dc1-d44c-4f78-98e9-10bed5a258b7 req-97a97174-4dc6-4cad-9fc7-fd3321cde257 service nova] Acquired lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:37:20 user nova-compute[71317]: DEBUG nova.network.neutron [req-7e970dc1-d44c-4f78-98e9-10bed5a258b7 req-97a97174-4dc6-4cad-9fc7-fd3321cde257 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Refreshing network info cache for port f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:37:20 user nova-compute[71317]: DEBUG nova.network.neutron [req-7e970dc1-d44c-4f78-98e9-10bed5a258b7 req-97a97174-4dc6-4cad-9fc7-fd3321cde257 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updated VIF entry in instance network info cache for port f204cf26-ba59-48a1-b08e-6d8915075961. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:37:20 user nova-compute[71317]: DEBUG nova.network.neutron [req-7e970dc1-d44c-4f78-98e9-10bed5a258b7 req-97a97174-4dc6-4cad-9fc7-fd3321cde257 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updating instance_info_cache with network_info: [{"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:37:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7e970dc1-d44c-4f78-98e9-10bed5a258b7 req-97a97174-4dc6-4cad-9fc7-fd3321cde257 service nova] Releasing lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:37:22 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Creating image(s)
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Ensure instance console log exists: /opt/stack/data/nova/instances/49c7ae26-8d9e-45ac-a189-3ee84e015b27/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Start _get_guest_xml network_info=[{"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tLxSpwaP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '49c7ae26-8d9e-45ac-a189-3ee84e015b27', 'attached_at': '', 'detached_at': '', 'volume_id': '495f173c-6873-42b6-a0fe-42da67f3aa75', 'serial': '495f173c-6873-42b6-a0fe-42da67f3aa75'}, 'attachment_id': '5a96efd1-e24e-41bb-aed3-6469de3a07a1', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:37:22 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:37:22 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tLxSpwaP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:37:22 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:37:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:37:22 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:37:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:37:22 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e9a7e2c-258d-4341-9d23-d258702aeff6]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:37:22 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[721b70b3-bcae-4a81-b10b-2558b4527b20]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75] added
Apr 14 04:37:22 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[30453597-e667-411a-9da1-9bf72021349f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:37:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ae49740-5844-400e-bef4-90cf2abc7da8]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tLxSpwaP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tLxSpwaP" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e0a72c1-15b8-423b-865f-27c46144d2da]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'tLxSpwaP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[17a87b2b-c283-4778-af78-c5a8d7059d9c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b126fecb-5f19-481c-bb33-91525687db55]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --login" returned: 0 in 0.063s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4f67262-3661-4089-97ec-b6c34addff05]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260]
Apr 14 04:37:23 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260] successful.
Apr 14 04:37:23 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d78e90f-c752-4460-9a15-8fe5c3bb94fd]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[608f152e-d386-45a3-a173-9d4749b8aa72]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:37:23 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.292s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] HCTL ('37', '-', '-', 0) found on session 50 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cf308d6e-920f-42be-b9b8-3a6cdc64ed62]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Searching for a device in session 50 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tLxSpwaP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:37:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.324s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== connect_volume: return (1326ms) {'type': 'block', 'scsi_wwn': '23132643462623363', 'path': '/dev/sdf'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23132643462623363', 'path': '/dev/sdf'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:37:17Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1983263653',display_name='tempest-TestVolumeBootPattern-server-1983263653',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1983263653',id=41,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFZO4o4kQgpEvaFt8iKOk4kfW7YF9jZ6XS2jVS2UAmaxmR76rP/jf7a3pf4xOyQu5V8HIaLR//SNI2rVtnFikRInNPtRWTjIBvrk9d4sXMIgR6FP64tMb+FLBXcO2su56A==',key_name='tempest-TestVolumeBootPattern-1640497739',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-mzzgtbac',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:37:18Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=49c7ae26-8d9e-45ac-a189-3ee84e015b27,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ff:ad:63,bridge_name='br-int',has_traffic_filtering=True,id=f204cf26-ba59-48a1-b08e-6d8915075961,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf204cf26-ba') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG nova.objects.instance [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lazy-loading 'pci_devices' on Instance uuid 49c7ae26-8d9e-45ac-a189-3ee84e015b27 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:37:24 user nova-compute[71317]:   <uuid>49c7ae26-8d9e-45ac-a189-3ee84e015b27</uuid>
Apr 14 04:37:24 user nova-compute[71317]:   <name>instance-00000029</name>
Apr 14 04:37:24 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:37:24 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:37:24 user nova-compute[71317]:   <metadata>
Apr 14 04:37:24 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:37:24 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:37:24 user nova-compute[71317]:       <nova:name>tempest-TestVolumeBootPattern-server-1983263653</nova:name>
Apr 14 04:37:24 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:37:22</nova:creationTime>
Apr 14 04:37:24 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:37:24 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:37:24 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:37:24 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:37:24 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:37:24 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:37:24 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:37:24 user nova-compute[71317]:       <nova:owner>
Apr 14 04:37:24 user nova-compute[71317]:         <nova:user uuid="bcf79de7da27495dbfd14c556b5b54c7">tempest-TestVolumeBootPattern-1780160917-project-member</nova:user>
Apr 14 04:37:24 user nova-compute[71317]:         <nova:project uuid="693d95858c624feaa60114aa89082186">tempest-TestVolumeBootPattern-1780160917</nova:project>
Apr 14 04:37:24 user nova-compute[71317]:       </nova:owner>
Apr 14 04:37:24 user nova-compute[71317]:       <nova:ports>
Apr 14 04:37:24 user nova-compute[71317]:         <nova:port uuid="f204cf26-ba59-48a1-b08e-6d8915075961">
Apr 14 04:37:24 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 04:37:24 user nova-compute[71317]:         </nova:port>
Apr 14 04:37:24 user nova-compute[71317]:       </nova:ports>
Apr 14 04:37:24 user nova-compute[71317]:     </nova:instance>
Apr 14 04:37:24 user nova-compute[71317]:   </metadata>
Apr 14 04:37:24 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:37:24 user nova-compute[71317]:     <system>
Apr 14 04:37:24 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:37:24 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:37:24 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:37:24 user nova-compute[71317]:       <entry name="serial">49c7ae26-8d9e-45ac-a189-3ee84e015b27</entry>
Apr 14 04:37:24 user nova-compute[71317]:       <entry name="uuid">49c7ae26-8d9e-45ac-a189-3ee84e015b27</entry>
Apr 14 04:37:24 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:37:24 user nova-compute[71317]:     </system>
Apr 14 04:37:24 user nova-compute[71317]:   </sysinfo>
Apr 14 04:37:24 user nova-compute[71317]:   <os>
Apr 14 04:37:24 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:37:24 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:37:24 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:37:24 user nova-compute[71317]:   </os>
Apr 14 04:37:24 user nova-compute[71317]:   <features>
Apr 14 04:37:24 user nova-compute[71317]:     <acpi/>
Apr 14 04:37:24 user nova-compute[71317]:     <apic/>
Apr 14 04:37:24 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:37:24 user nova-compute[71317]:   </features>
Apr 14 04:37:24 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:37:24 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:37:24 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:37:24 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:37:24 user nova-compute[71317]:   </clock>
Apr 14 04:37:24 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:37:24 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:37:24 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:37:24 user nova-compute[71317]:   </cpu>
Apr 14 04:37:24 user nova-compute[71317]:   <devices>
Apr 14 04:37:24 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:37:24 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:37:24 user nova-compute[71317]:       <source dev="/dev/sdf"/>
Apr 14 04:37:24 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:37:24 user nova-compute[71317]:       <serial>495f173c-6873-42b6-a0fe-42da67f3aa75</serial>
Apr 14 04:37:24 user nova-compute[71317]:     </disk>
Apr 14 04:37:24 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:37:24 user nova-compute[71317]:       <mac address="fa:16:3e:ff:ad:63"/>
Apr 14 04:37:24 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:37:24 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:37:24 user nova-compute[71317]:       <target dev="tapf204cf26-ba"/>
Apr 14 04:37:24 user nova-compute[71317]:     </interface>
Apr 14 04:37:24 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:37:24 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/49c7ae26-8d9e-45ac-a189-3ee84e015b27/console.log" append="off"/>
Apr 14 04:37:24 user nova-compute[71317]:     </serial>
Apr 14 04:37:24 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:37:24 user nova-compute[71317]:     <video>
Apr 14 04:37:24 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:37:24 user nova-compute[71317]:     </video>
Apr 14 04:37:24 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:37:24 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:37:24 user nova-compute[71317]:     </rng>
Apr 14 04:37:24 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:37:24 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:37:24 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:37:24 user nova-compute[71317]:     </memballoon>
Apr 14 04:37:24 user nova-compute[71317]:   </devices>
Apr 14 04:37:24 user nova-compute[71317]: </domain>
Apr 14 04:37:24 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:37:17Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1983263653',display_name='tempest-TestVolumeBootPattern-server-1983263653',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1983263653',id=41,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFZO4o4kQgpEvaFt8iKOk4kfW7YF9jZ6XS2jVS2UAmaxmR76rP/jf7a3pf4xOyQu5V8HIaLR//SNI2rVtnFikRInNPtRWTjIBvrk9d4sXMIgR6FP64tMb+FLBXcO2su56A==',key_name='tempest-TestVolumeBootPattern-1640497739',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-mzzgtbac',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:37:18Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=49c7ae26-8d9e-45ac-a189-3ee84e015b27,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ff:ad:63,bridge_name='br-int',has_traffic_filtering=True,id=f204cf26-ba59-48a1-b08e-6d8915075961,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf204cf26-ba') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG os_vif [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ff:ad:63,bridge_name='br-int',has_traffic_filtering=True,id=f204cf26-ba59-48a1-b08e-6d8915075961,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf204cf26-ba') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapf204cf26-ba, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapf204cf26-ba, col_values=(('external_ids', {'iface-id': 'f204cf26-ba59-48a1-b08e-6d8915075961', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ff:ad:63', 'vm-uuid': '49c7ae26-8d9e-45ac-a189-3ee84e015b27'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:24 user nova-compute[71317]: INFO os_vif [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ff:ad:63,bridge_name='br-int',has_traffic_filtering=True,id=f204cf26-ba59-48a1-b08e-6d8915075961,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf204cf26-ba')
Apr 14 04:37:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:37:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] No VIF found with MAC fa:16:3e:ff:ad:63, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:37:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-404ac653-e345-461c-87bf-ac9418fab40e req-4d6c3b51-4975-4007-af6d-0cb1e92dfe50 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Received event network-vif-plugged-f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:37:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-404ac653-e345-461c-87bf-ac9418fab40e req-4d6c3b51-4975-4007-af6d-0cb1e92dfe50 service nova] Acquiring lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:37:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-404ac653-e345-461c-87bf-ac9418fab40e req-4d6c3b51-4975-4007-af6d-0cb1e92dfe50 service nova] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:37:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-404ac653-e345-461c-87bf-ac9418fab40e req-4d6c3b51-4975-4007-af6d-0cb1e92dfe50 service nova] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:37:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-404ac653-e345-461c-87bf-ac9418fab40e req-4d6c3b51-4975-4007-af6d-0cb1e92dfe50 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] No waiting events found dispatching network-vif-plugged-f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:37:25 user nova-compute[71317]: WARNING nova.compute.manager [req-404ac653-e345-461c-87bf-ac9418fab40e req-4d6c3b51-4975-4007-af6d-0cb1e92dfe50 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Received unexpected event network-vif-plugged-f204cf26-ba59-48a1-b08e-6d8915075961 for instance with vm_state building and task_state spawning.
Apr 14 04:37:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447047.7113016, 49c7ae26-8d9e-45ac-a189-3ee84e015b27 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:37:27 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] VM Resumed (Lifecycle Event)
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:37:27 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Instance spawned successfully.
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:37:27 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447047.712737, 49c7ae26-8d9e-45ac-a189-3ee84e015b27 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:37:27 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] VM Started (Lifecycle Event)
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-f28e61fe-919b-499c-91d5-c609e6606023 req-36bded07-e7e7-44f1-af59-fe68524b7d5e service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Received event network-vif-plugged-f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f28e61fe-919b-499c-91d5-c609e6606023 req-36bded07-e7e7-44f1-af59-fe68524b7d5e service nova] Acquiring lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f28e61fe-919b-499c-91d5-c609e6606023 req-36bded07-e7e7-44f1-af59-fe68524b7d5e service nova] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f28e61fe-919b-499c-91d5-c609e6606023 req-36bded07-e7e7-44f1-af59-fe68524b7d5e service nova] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-f28e61fe-919b-499c-91d5-c609e6606023 req-36bded07-e7e7-44f1-af59-fe68524b7d5e service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] No waiting events found dispatching network-vif-plugged-f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:37:27 user nova-compute[71317]: WARNING nova.compute.manager [req-f28e61fe-919b-499c-91d5-c609e6606023 req-36bded07-e7e7-44f1-af59-fe68524b7d5e service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Received unexpected event network-vif-plugged-f204cf26-ba59-48a1-b08e-6d8915075961 for instance with vm_state building and task_state spawning.
Apr 14 04:37:27 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:37:27 user nova-compute[71317]: INFO nova.compute.manager [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Took 5.06 seconds to spawn the instance on the hypervisor.
Apr 14 04:37:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:37:27 user nova-compute[71317]: INFO nova.compute.manager [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Took 10.36 seconds to build instance.
Apr 14 04:37:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7572834d-6bac-47ce-b3d7-0fe44ce0c9c9 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.461s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:37:28 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447033.1875515, 390ca845-5660-49d0-bd8a-fcfc39176ca5 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:37:28 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] VM Stopped (Lifecycle Event)
Apr 14 04:37:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5a11ac3c-f4c9-4370-8394-55b2846a8f9b None None] [instance: 390ca845-5660-49d0-bd8a-fcfc39176ca5] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:37:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:35 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:37:37 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:37:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:37:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:37:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:37:37 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:37:37 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Updating instance_info_cache with network_info: [{"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:37:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:37:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:37:37 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:37:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:40 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:37:43 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:37:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:44 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:37:45 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:37:46 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:37:46 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=7899MB free_disk=26.150794982910156GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 19a166de-f0c9-43f2-8c88-585e0d5d22cc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 970e81d2-c227-4eab-92dd-1ed732e67968 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 283ef25b-314a-47c1-8414-9d7c737e5115 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 695d931b-1f30-4d57-b6d7-d199d2874e65 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 49c7ae26-8d9e-45ac-a189-3ee84e015b27 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1728MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing inventories for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating ProviderTree inventory for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating inventory in ProviderTree for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing aggregate associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, aggregates: None {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing trait associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, traits: COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSE41,HW_CPU_X86_SSE42,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_TRUSTED_CERTS,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_ACCELERATORS,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NET_VIF_MODEL_E1000,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_NODE,HW_CPU_X86_SSE2,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,HW_CPU_X86_SSSE3,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_STORAGE_BUS_USB,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_NE2K_PCI {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.637s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 04:37:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] There are 0 instances to clean {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 04:37:47 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:37:47 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:37:47 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:37:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:37:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:37:56 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:37:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances with incomplete migration  {{(pid=71317) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 04:37:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:38:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:38:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "970e81d2-c227-4eab-92dd-1ed732e67968" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:38:21 user nova-compute[71317]: INFO nova.compute.manager [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Terminating instance
Apr 14 04:38:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-55bc72bf-8493-4f1f-a3c1-6b89fb760eb5 req-d5045adb-b1f5-40bd-bd64-9c3038956a36 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-unplugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-55bc72bf-8493-4f1f-a3c1-6b89fb760eb5 req-d5045adb-b1f5-40bd-bd64-9c3038956a36 service nova] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-55bc72bf-8493-4f1f-a3c1-6b89fb760eb5 req-d5045adb-b1f5-40bd-bd64-9c3038956a36 service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-55bc72bf-8493-4f1f-a3c1-6b89fb760eb5 req-d5045adb-b1f5-40bd-bd64-9c3038956a36 service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-55bc72bf-8493-4f1f-a3c1-6b89fb760eb5 req-d5045adb-b1f5-40bd-bd64-9c3038956a36 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] No waiting events found dispatching network-vif-unplugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:38:21 user nova-compute[71317]: DEBUG nova.compute.manager [req-55bc72bf-8493-4f1f-a3c1-6b89fb760eb5 req-d5045adb-b1f5-40bd-bd64-9c3038956a36 service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-unplugged-45a5a643-510a-45e2-a8e4-12eb53631daf for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:38:22 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Instance destroyed successfully.
Apr 14 04:38:22 user nova-compute[71317]: DEBUG nova.objects.instance [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'resources' on Instance uuid 970e81d2-c227-4eab-92dd-1ed732e67968 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:32:48Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1167245313',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1167245313',id=35,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:34:48Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-djyaua3x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:36:36Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=970e81d2-c227-4eab-92dd-1ed732e67968,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "45a5a643-510a-45e2-a8e4-12eb53631daf", "address": "fa:16:3e:cd:43:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap45a5a643-51", "ovs_interfaceid": "45a5a643-510a-45e2-a8e4-12eb53631daf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:cd:43:7e,bridge_name='br-int',has_traffic_filtering=True,id=45a5a643-510a-45e2-a8e4-12eb53631daf,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap45a5a643-51') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_vif [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:cd:43:7e,bridge_name='br-int',has_traffic_filtering=True,id=45a5a643-510a-45e2-a8e4-12eb53631daf,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap45a5a643-51') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap45a5a643-51, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:22 user nova-compute[71317]: INFO os_vif [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:cd:43:7e,bridge_name='br-int',has_traffic_filtering=True,id=45a5a643-510a-45e2-a8e4-12eb53631daf,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap45a5a643-51')
Apr 14 04:38:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'deafxyMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:38:22 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[918cba85-c218-48ae-b327-3afbed32f469]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:38:22 user nova-compute[71317]: No targets found.
Apr 14 04:38:22 user nova-compute[71317]: iSNS:
Apr 14 04:38:22 user nova-compute[71317]: No targets found.
Apr 14 04:38:22 user nova-compute[71317]: STATIC:
Apr 14 04:38:22 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757
Apr 14 04:38:22 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:38:22 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:38:22 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269
Apr 14 04:38:22 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:38:22 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:38:22 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970
Apr 14 04:38:22 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:38:22 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:38:22 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75
Apr 14 04:38:22 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:38:22 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:38:22 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:38:22 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:38:22 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:38:22 user nova-compute[71317]: FIRMWARE:
Apr 14 04:38:22 user nova-compute[71317]: No targets found.
Apr 14 04:38:22 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:38:22 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:38:22 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[02214925-642a-4997-8c99-5be100c0d9a4]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1923057-8778-4698-b5af-25abd152eeb0]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:38:22 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:38:22 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:38:22 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:38:22 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:38:22 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:38:22 user nova-compute[71317]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 (non-flash)
Apr 14 04:38:22 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:38:22 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:38:22 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:38:22 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84b41355a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b3f13dc-58ed-42df-accd-4a39e8e8c3b3]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a51ad799-d776-4728-8a40-76dcf4f60787]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1084d195-1708-4856-beaf-efd148ed4884]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.041s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a67bb4e-2a89-4a3f-8163-49a33d6cf3e8]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-SSCST_BIO_Storage_aa5fc129', '/dev/disk/by-id/scsi-26334313830363633', '/dev/disk/by-id/scsi-1SCST_BIOaa5fc129-c418066358a6641a') {{(pid=81937) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6c6e1ed-3317-495c-8dbf-859b333c5ae6]: (4, None) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[390fd17c-2b40-4a52-81b3-199116eae323]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --logout" returned: 0 in 0.065s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[85b9aa11-b7d4-4b77-b319-cc23898ad8b7]: (4, ('Logging out of session [sid: 44, target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970, portal: 172.16.0.220,3260]\nLogout of [sid: 44, target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 44, target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970, portal: 172.16.0.220,3260]
Apr 14 04:38:22 user nova-compute[71317]: Logout of [sid: 44, target: iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970, portal: 172.16.0.220,3260] successful.
Apr 14 04:38:22 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9108333c-dbbf-4edb-a2b5-c6f034073970 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c26fb2f-6c4e-4f21-bada-e27ccf9c9c23]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.343s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] <== disconnect_volume: return (345ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:38:22 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Deleting instance files /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968_del
Apr 14 04:38:22 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Deletion of /opt/stack/data/nova/instances/970e81d2-c227-4eab-92dd-1ed732e67968_del complete
Apr 14 04:38:22 user nova-compute[71317]: INFO nova.compute.manager [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Took 1.21 seconds to destroy the instance on the hypervisor.
Apr 14 04:38:22 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:38:22 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:38:23 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:38:23 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Took 0.47 seconds to deallocate network for instance.
Apr 14 04:38:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-7c70633f-bc9d-4acb-9697-1e15d1068cb8 req-86050a4b-ebe9-4af3-b3dc-5c81a7fd923d service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-deleted-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:38:23 user nova-compute[71317]: INFO nova.compute.manager [req-7c70633f-bc9d-4acb-9697-1e15d1068cb8 req-86050a4b-ebe9-4af3-b3dc-5c81a7fd923d service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Neutron deleted interface 45a5a643-510a-45e2-a8e4-12eb53631daf; detaching it from the instance and deleting it from the info cache
Apr 14 04:38:23 user nova-compute[71317]: DEBUG nova.network.neutron [req-7c70633f-bc9d-4acb-9697-1e15d1068cb8 req-86050a4b-ebe9-4af3-b3dc-5c81a7fd923d service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:38:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-7c70633f-bc9d-4acb-9697-1e15d1068cb8 req-86050a4b-ebe9-4af3-b3dc-5c81a7fd923d service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Detach interface failed, port_id=45a5a643-510a-45e2-a8e4-12eb53631daf, reason: Instance 970e81d2-c227-4eab-92dd-1ed732e67968 could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:38:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-4917ffe9-9808-4cda-9f31-babe8281d21b req-8d20b9de-9c84-4639-9919-98e34e98887a service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:38:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4917ffe9-9808-4cda-9f31-babe8281d21b req-8d20b9de-9c84-4639-9919-98e34e98887a service nova] Acquiring lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:38:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4917ffe9-9808-4cda-9f31-babe8281d21b req-8d20b9de-9c84-4639-9919-98e34e98887a service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:38:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4917ffe9-9808-4cda-9f31-babe8281d21b req-8d20b9de-9c84-4639-9919-98e34e98887a service nova] Lock "970e81d2-c227-4eab-92dd-1ed732e67968-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:38:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-4917ffe9-9808-4cda-9f31-babe8281d21b req-8d20b9de-9c84-4639-9919-98e34e98887a service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] No waiting events found dispatching network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:38:23 user nova-compute[71317]: WARNING nova.compute.manager [req-4917ffe9-9808-4cda-9f31-babe8281d21b req-8d20b9de-9c84-4639-9919-98e34e98887a service nova] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Received unexpected event network-vif-plugged-45a5a643-510a-45e2-a8e4-12eb53631daf for instance with vm_state active and task_state deleting.
Apr 14 04:38:24 user nova-compute[71317]: INFO nova.compute.manager [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Took 0.83 seconds to detach 1 volumes for instance.
Apr 14 04:38:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:38:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:38:24 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:38:24 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:38:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.313s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:38:24 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Deleted allocations for instance 970e81d2-c227-4eab-92dd-1ed732e67968
Apr 14 04:38:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-295f03cf-81f8-4e44-b8e8-34a4f61ee3a4 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "970e81d2-c227-4eab-92dd-1ed732e67968" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:38:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:36 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:38:37 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447102.3066297, 970e81d2-c227-4eab-92dd-1ed732e67968 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:38:37 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] VM Stopped (Lifecycle Event)
Apr 14 04:38:37 user nova-compute[71317]: DEBUG nova.compute.manager [None req-df26228e-bffc-47c9-b362-88de4987795d None None] [instance: 970e81d2-c227-4eab-92dd-1ed732e67968] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:38:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:38 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:38:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:39 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:38:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:38:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:38:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:38:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:38:39 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:38:39 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lazy-loading 'info_cache' on Instance uuid 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:38:39 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Updating instance_info_cache with network_info: [{"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:38:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-226b0e81-b6a2-4094-b0f1-3fb0d3b94074" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:38:39 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:38:40 user nova-compute[71317]: INFO nova.compute.manager [None req-894064b2-6948-428b-aa97-c1bcd7ffe895 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Get console output
Apr 14 04:38:40 user nova-compute[71317]: INFO nova.privsep.libvirt [-] Ignored error while reading from instance console pty: can't concat NoneType to bytes
Apr 14 04:38:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-da0a8366-d8b8-403d-a6b8-7603fb83ee21 req-e8aa7062-5e0e-436f-90e1-b5fe692ddb7b service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Received event network-changed-345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:38:40 user nova-compute[71317]: DEBUG nova.compute.manager [req-da0a8366-d8b8-403d-a6b8-7603fb83ee21 req-e8aa7062-5e0e-436f-90e1-b5fe692ddb7b service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Refreshing instance network info cache due to event network-changed-345c5341-edef-476a-9b94-e229dc1594b8. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:38:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-da0a8366-d8b8-403d-a6b8-7603fb83ee21 req-e8aa7062-5e0e-436f-90e1-b5fe692ddb7b service nova] Acquiring lock "refresh_cache-6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:38:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-da0a8366-d8b8-403d-a6b8-7603fb83ee21 req-e8aa7062-5e0e-436f-90e1-b5fe692ddb7b service nova] Acquired lock "refresh_cache-6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:38:40 user nova-compute[71317]: DEBUG nova.network.neutron [req-da0a8366-d8b8-403d-a6b8-7603fb83ee21 req-e8aa7062-5e0e-436f-90e1-b5fe692ddb7b service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Refreshing network info cache for port 345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:38:41 user nova-compute[71317]: INFO nova.compute.manager [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Terminating instance
Apr 14 04:38:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG nova.network.neutron [req-da0a8366-d8b8-403d-a6b8-7603fb83ee21 req-e8aa7062-5e0e-436f-90e1-b5fe692ddb7b service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Updated VIF entry in instance network info cache for port 345c5341-edef-476a-9b94-e229dc1594b8. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG nova.network.neutron [req-da0a8366-d8b8-403d-a6b8-7603fb83ee21 req-e8aa7062-5e0e-436f-90e1-b5fe692ddb7b service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Updating instance_info_cache with network_info: [{"id": "345c5341-edef-476a-9b94-e229dc1594b8", "address": "fa:16:3e:32:e2:76", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.198", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap345c5341-ed", "ovs_interfaceid": "345c5341-edef-476a-9b94-e229dc1594b8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-da0a8366-d8b8-403d-a6b8-7603fb83ee21 req-e8aa7062-5e0e-436f-90e1-b5fe692ddb7b service nova] Releasing lock "refresh_cache-6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:38:41 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Instance destroyed successfully.
Apr 14 04:38:41 user nova-compute[71317]: DEBUG nova.objects.instance [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'resources' on Instance uuid 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:36:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1738062211',display_name='tempest-AttachVolumeNegativeTest-server-1738062211',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1738062211',id=40,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAesg3BacL1BvdUPXS1xDD/3pqX31IW3Mm7TfS7lU8AJ9AncfmFmDbzZwUcbDyrA39s0koYymJjsrRF/BTnfmAyYfXAFKaGrYBH4j+YINr5ySxpEfDWQcwUiaidQx4WbYw==',key_name='tempest-keypair-2114462517',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:36:56Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='601e2806907a48d3995cc90b54da3c8c',ramdisk_id='',reservation_id='r-r5rv1iee',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-1197484356',owner_user_name='tempest-AttachVolumeNegativeTest-1197484356-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:36:57Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7cab2461ab0d4eb08b62feaf6396bc87',uuid=6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "345c5341-edef-476a-9b94-e229dc1594b8", "address": "fa:16:3e:32:e2:76", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap345c5341-ed", "ovs_interfaceid": "345c5341-edef-476a-9b94-e229dc1594b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converting VIF {"id": "345c5341-edef-476a-9b94-e229dc1594b8", "address": "fa:16:3e:32:e2:76", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap345c5341-ed", "ovs_interfaceid": "345c5341-edef-476a-9b94-e229dc1594b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:32:e2:76,bridge_name='br-int',has_traffic_filtering=True,id=345c5341-edef-476a-9b94-e229dc1594b8,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap345c5341-ed') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG os_vif [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:32:e2:76,bridge_name='br-int',has_traffic_filtering=True,id=345c5341-edef-476a-9b94-e229dc1594b8,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap345c5341-ed') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap345c5341-ed, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:41 user nova-compute[71317]: INFO os_vif [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:32:e2:76,bridge_name='br-int',has_traffic_filtering=True,id=345c5341-edef-476a-9b94-e229dc1594b8,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap345c5341-ed')
Apr 14 04:38:41 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Deleting instance files /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6_del
Apr 14 04:38:41 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Deletion of /opt/stack/data/nova/instances/6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6_del complete
Apr 14 04:38:41 user nova-compute[71317]: INFO nova.compute.manager [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Took 0.79 seconds to destroy the instance on the hypervisor.
Apr 14 04:38:41 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:38:41 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:38:42 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:38:42 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Took 0.93 seconds to deallocate network for instance.
Apr 14 04:38:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-c056bb60-ef24-45b6-be4c-c700e56793b1 req-7f5ae7dd-fd9c-4a3c-b30f-4c3e71cabee1 service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Received event network-vif-deleted-345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:38:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:38:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:38:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-4fe66287-d9a6-4805-bd8f-2baf17260e4e req-cf8e3ba6-8497-4572-9a5e-e4386cc9caef service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Received event network-vif-unplugged-345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:38:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4fe66287-d9a6-4805-bd8f-2baf17260e4e req-cf8e3ba6-8497-4572-9a5e-e4386cc9caef service nova] Acquiring lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:38:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4fe66287-d9a6-4805-bd8f-2baf17260e4e req-cf8e3ba6-8497-4572-9a5e-e4386cc9caef service nova] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:38:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4fe66287-d9a6-4805-bd8f-2baf17260e4e req-cf8e3ba6-8497-4572-9a5e-e4386cc9caef service nova] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:38:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-4fe66287-d9a6-4805-bd8f-2baf17260e4e req-cf8e3ba6-8497-4572-9a5e-e4386cc9caef service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] No waiting events found dispatching network-vif-unplugged-345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:38:42 user nova-compute[71317]: WARNING nova.compute.manager [req-4fe66287-d9a6-4805-bd8f-2baf17260e4e req-cf8e3ba6-8497-4572-9a5e-e4386cc9caef service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Received unexpected event network-vif-unplugged-345c5341-edef-476a-9b94-e229dc1594b8 for instance with vm_state deleted and task_state None.
Apr 14 04:38:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-4fe66287-d9a6-4805-bd8f-2baf17260e4e req-cf8e3ba6-8497-4572-9a5e-e4386cc9caef service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Received event network-vif-plugged-345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:38:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4fe66287-d9a6-4805-bd8f-2baf17260e4e req-cf8e3ba6-8497-4572-9a5e-e4386cc9caef service nova] Acquiring lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:38:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4fe66287-d9a6-4805-bd8f-2baf17260e4e req-cf8e3ba6-8497-4572-9a5e-e4386cc9caef service nova] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:38:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4fe66287-d9a6-4805-bd8f-2baf17260e4e req-cf8e3ba6-8497-4572-9a5e-e4386cc9caef service nova] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:38:42 user nova-compute[71317]: DEBUG nova.compute.manager [req-4fe66287-d9a6-4805-bd8f-2baf17260e4e req-cf8e3ba6-8497-4572-9a5e-e4386cc9caef service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] No waiting events found dispatching network-vif-plugged-345c5341-edef-476a-9b94-e229dc1594b8 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:38:42 user nova-compute[71317]: WARNING nova.compute.manager [req-4fe66287-d9a6-4805-bd8f-2baf17260e4e req-cf8e3ba6-8497-4572-9a5e-e4386cc9caef service nova] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Received unexpected event network-vif-plugged-345c5341-edef-476a-9b94-e229dc1594b8 for instance with vm_state deleted and task_state None.
Apr 14 04:38:43 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:38:43 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:38:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.267s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:38:43 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Deleted allocations for instance 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6
Apr 14 04:38:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-7caa0ba7-2a9a-4238-ae2e-f08b99c43953 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.160s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:38:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:38:46 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:38:47 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:38:47 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:38:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8227MB free_disk=26.167896270751953GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 19a166de-f0c9-43f2-8c88-585e0d5d22cc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 283ef25b-314a-47c1-8414-9d7c737e5115 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 695d931b-1f30-4d57-b6d7-d199d2874e65 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 49c7ae26-8d9e-45ac-a189-3ee84e015b27 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 7 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:38:47 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1472MB phys_disk=40GB used_disk=4GB total_vcpus=12 used_vcpus=7 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:38:48 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:38:48 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:38:48 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:38:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.352s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:38:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:38:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:38:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:38:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:38:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:38:56 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447121.7611609, 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:38:56 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] VM Stopped (Lifecycle Event)
Apr 14 04:38:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-91810c29-9f33-4e7b-b5a4-1918664dc65f None None] [instance: 6a8a0a2f-c4a1-4ee1-a132-0b79bf1f29b6] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:38:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:38:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-0e7f1daa-e185-4ad0-a2e1-3c4dc26a18e1 req-1076aa86-3792-4728-ae92-267d78e1ec37 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Received event network-changed-f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:39:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-0e7f1daa-e185-4ad0-a2e1-3c4dc26a18e1 req-1076aa86-3792-4728-ae92-267d78e1ec37 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Refreshing instance network info cache due to event network-changed-f204cf26-ba59-48a1-b08e-6d8915075961. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:39:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0e7f1daa-e185-4ad0-a2e1-3c4dc26a18e1 req-1076aa86-3792-4728-ae92-267d78e1ec37 service nova] Acquiring lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:39:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0e7f1daa-e185-4ad0-a2e1-3c4dc26a18e1 req-1076aa86-3792-4728-ae92-267d78e1ec37 service nova] Acquired lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:39:10 user nova-compute[71317]: DEBUG nova.network.neutron [req-0e7f1daa-e185-4ad0-a2e1-3c4dc26a18e1 req-1076aa86-3792-4728-ae92-267d78e1ec37 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Refreshing network info cache for port f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:39:11 user nova-compute[71317]: DEBUG nova.network.neutron [req-0e7f1daa-e185-4ad0-a2e1-3c4dc26a18e1 req-1076aa86-3792-4728-ae92-267d78e1ec37 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updated VIF entry in instance network info cache for port f204cf26-ba59-48a1-b08e-6d8915075961. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:39:11 user nova-compute[71317]: DEBUG nova.network.neutron [req-0e7f1daa-e185-4ad0-a2e1-3c4dc26a18e1 req-1076aa86-3792-4728-ae92-267d78e1ec37 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updating instance_info_cache with network_info: [{"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.171", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:39:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0e7f1daa-e185-4ad0-a2e1-3c4dc26a18e1 req-1076aa86-3792-4728-ae92-267d78e1ec37 service nova] Releasing lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:39:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:12 user nova-compute[71317]: INFO nova.compute.manager [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Terminating instance
Apr 14 04:39:12 user nova-compute[71317]: DEBUG nova.compute.manager [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-f0442e61-549b-4bc2-8538-ea9982155d16 req-b5fb0fe6-d360-4e34-b368-d134ebd7cbf9 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Received event network-vif-unplugged-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0442e61-549b-4bc2-8538-ea9982155d16 req-b5fb0fe6-d360-4e34-b368-d134ebd7cbf9 service nova] Acquiring lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0442e61-549b-4bc2-8538-ea9982155d16 req-b5fb0fe6-d360-4e34-b368-d134ebd7cbf9 service nova] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0442e61-549b-4bc2-8538-ea9982155d16 req-b5fb0fe6-d360-4e34-b368-d134ebd7cbf9 service nova] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-f0442e61-549b-4bc2-8538-ea9982155d16 req-b5fb0fe6-d360-4e34-b368-d134ebd7cbf9 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] No waiting events found dispatching network-vif-unplugged-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-f0442e61-549b-4bc2-8538-ea9982155d16 req-b5fb0fe6-d360-4e34-b368-d134ebd7cbf9 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Received event network-vif-unplugged-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Instance destroyed successfully.
Apr 14 04:39:12 user nova-compute[71317]: DEBUG nova.objects.instance [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'resources' on Instance uuid 19a166de-f0c9-43f2-8c88-585e0d5d22cc {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:29:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-717082241',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-717082241',id=31,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:29:56Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-tuh7ljtq',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:31:46Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=19a166de-f0c9-43f2-8c88-585e0d5d22cc,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "address": "fa:16:3e:b6:df:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape4576e1c-df", "ovs_interfaceid": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "address": "fa:16:3e:b6:df:7e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape4576e1c-df", "ovs_interfaceid": "e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b6:df:7e,bridge_name='br-int',has_traffic_filtering=True,id=e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape4576e1c-df') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG os_vif [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:b6:df:7e,bridge_name='br-int',has_traffic_filtering=True,id=e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape4576e1c-df') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape4576e1c-df, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:39:12 user nova-compute[71317]: INFO os_vif [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:b6:df:7e,bridge_name='br-int',has_traffic_filtering=True,id=e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape4576e1c-df')
Apr 14 04:39:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Deleting instance files /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc_del
Apr 14 04:39:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Deletion of /opt/stack/data/nova/instances/19a166de-f0c9-43f2-8c88-585e0d5d22cc_del complete
Apr 14 04:39:13 user nova-compute[71317]: INFO nova.compute.manager [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Took 0.77 seconds to destroy the instance on the hypervisor.
Apr 14 04:39:13 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:39:13 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:39:13 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:39:13 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:39:13 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Took 0.49 seconds to deallocate network for instance.
Apr 14 04:39:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-bd2fb9cd-d1e9-4b8c-916f-7a5ce781dde7 req-52172413-df33-491f-be16-b5bcc0949cc3 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Received event network-vif-deleted-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:39:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:13 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:39:13 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:39:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.248s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:13 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Deleted allocations for instance 19a166de-f0c9-43f2-8c88-585e0d5d22cc
Apr 14 04:39:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-1bc589e7-1a00-4861-b491-7109a7e87471 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.703s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-43f69a4b-5ea1-4e79-88b7-038020acb24e req-0443816d-7734-488e-a66a-994b7ff79d90 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Received event network-vif-plugged-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:39:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-43f69a4b-5ea1-4e79-88b7-038020acb24e req-0443816d-7734-488e-a66a-994b7ff79d90 service nova] Acquiring lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-43f69a4b-5ea1-4e79-88b7-038020acb24e req-0443816d-7734-488e-a66a-994b7ff79d90 service nova] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:14 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-43f69a4b-5ea1-4e79-88b7-038020acb24e req-0443816d-7734-488e-a66a-994b7ff79d90 service nova] Lock "19a166de-f0c9-43f2-8c88-585e0d5d22cc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:14 user nova-compute[71317]: DEBUG nova.compute.manager [req-43f69a4b-5ea1-4e79-88b7-038020acb24e req-0443816d-7734-488e-a66a-994b7ff79d90 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] No waiting events found dispatching network-vif-plugged-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:39:14 user nova-compute[71317]: WARNING nova.compute.manager [req-43f69a4b-5ea1-4e79-88b7-038020acb24e req-0443816d-7734-488e-a66a-994b7ff79d90 service nova] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Received unexpected event network-vif-plugged-e4576e1c-dfe3-4ab1-aa2a-4edfad4dd02d for instance with vm_state deleted and task_state None.
Apr 14 04:39:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "9d93dc7e-d497-4d8f-9ade-c508170f6121" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:39:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:39:18 user nova-compute[71317]: INFO nova.compute.claims [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Claim successful on node user
Apr 14 04:39:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:18 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:39:18 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:39:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.305s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:39:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:39:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:39:18 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:39:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:39:19 user nova-compute[71317]: INFO nova.virt.block_device [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Booting with volume 350da73c-f3b8-4777-b0d2-56bc054366d1 at /dev/vda
Apr 14 04:39:19 user nova-compute[71317]: DEBUG nova.policy [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bcf79de7da27495dbfd14c556b5b54c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '693d95858c624feaa60114aa89082186', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG os_brick.utils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecdedeb2-71e6-4a4c-b220-379740656bf9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[019b4afe-577b-4db8-87f4-1f42141ca5e0]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c875fec-2cc6-4ab1-8a3b-798931a8ddca]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a07cab4a-893f-4d24-b141-01a3f2a58df3]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:39:19 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:39:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2489ff44-774b-4655-9bbf-db0cbccf19a7]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG os_brick.utils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== get_connector_properties: return (115ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Updating existing volume attachment record: 858be4aa-e57b-41f7-baa4-4bb0174a0dbd {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:39:19 user nova-compute[71317]: DEBUG nova.network.neutron [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Successfully created port: 2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:39:20 user nova-compute[71317]: DEBUG nova.network.neutron [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Successfully updated port: 2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:39:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "refresh_cache-9d93dc7e-d497-4d8f-9ade-c508170f6121" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:39:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquired lock "refresh_cache-9d93dc7e-d497-4d8f-9ade-c508170f6121" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:39:20 user nova-compute[71317]: DEBUG nova.network.neutron [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:39:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-d10ec554-d724-4046-8305-4be30c0bfcd2 req-76dc1119-3619-4f79-a196-8646aabb857f service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Received event network-changed-2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:39:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-d10ec554-d724-4046-8305-4be30c0bfcd2 req-76dc1119-3619-4f79-a196-8646aabb857f service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Refreshing instance network info cache due to event network-changed-2fcf791e-22d7-4a96-894c-70900772a5ff. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:39:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d10ec554-d724-4046-8305-4be30c0bfcd2 req-76dc1119-3619-4f79-a196-8646aabb857f service nova] Acquiring lock "refresh_cache-9d93dc7e-d497-4d8f-9ade-c508170f6121" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:39:20 user nova-compute[71317]: DEBUG nova.network.neutron [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:39:21 user nova-compute[71317]: DEBUG nova.network.neutron [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Updating instance_info_cache with network_info: [{"id": "2fcf791e-22d7-4a96-894c-70900772a5ff", "address": "fa:16:3e:67:49:c3", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2fcf791e-22", "ovs_interfaceid": "2fcf791e-22d7-4a96-894c-70900772a5ff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:39:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Releasing lock "refresh_cache-9d93dc7e-d497-4d8f-9ade-c508170f6121" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:39:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Instance network_info: |[{"id": "2fcf791e-22d7-4a96-894c-70900772a5ff", "address": "fa:16:3e:67:49:c3", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2fcf791e-22", "ovs_interfaceid": "2fcf791e-22d7-4a96-894c-70900772a5ff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:39:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d10ec554-d724-4046-8305-4be30c0bfcd2 req-76dc1119-3619-4f79-a196-8646aabb857f service nova] Acquired lock "refresh_cache-9d93dc7e-d497-4d8f-9ade-c508170f6121" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:39:21 user nova-compute[71317]: DEBUG nova.network.neutron [req-d10ec554-d724-4046-8305-4be30c0bfcd2 req-76dc1119-3619-4f79-a196-8646aabb857f service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Refreshing network info cache for port 2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:39:21 user nova-compute[71317]: DEBUG nova.network.neutron [req-d10ec554-d724-4046-8305-4be30c0bfcd2 req-76dc1119-3619-4f79-a196-8646aabb857f service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Updated VIF entry in instance network info cache for port 2fcf791e-22d7-4a96-894c-70900772a5ff. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:39:21 user nova-compute[71317]: DEBUG nova.network.neutron [req-d10ec554-d724-4046-8305-4be30c0bfcd2 req-76dc1119-3619-4f79-a196-8646aabb857f service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Updating instance_info_cache with network_info: [{"id": "2fcf791e-22d7-4a96-894c-70900772a5ff", "address": "fa:16:3e:67:49:c3", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2fcf791e-22", "ovs_interfaceid": "2fcf791e-22d7-4a96-894c-70900772a5ff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:39:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-d10ec554-d724-4046-8305-4be30c0bfcd2 req-76dc1119-3619-4f79-a196-8646aabb857f service nova] Releasing lock "refresh_cache-9d93dc7e-d497-4d8f-9ade-c508170f6121" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:39:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:39:23 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Creating image(s)
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Did not create local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Ensure instance console log exists: /opt/stack/data/nova/instances/9d93dc7e-d497-4d8f-9ade-c508170f6121/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Start _get_guest_xml network_info=[{"id": "2fcf791e-22d7-4a96-894c-70900772a5ff", "address": "fa:16:3e:67:49:c3", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2fcf791e-22", "ovs_interfaceid": "2fcf791e-22d7-4a96-894c-70900772a5ff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': 0, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'utAhjSFl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '9d93dc7e-d497-4d8f-9ade-c508170f6121', 'attached_at': '', 'detached_at': '', 'volume_id': '350da73c-f3b8-4777-b0d2-56bc054366d1', 'serial': '350da73c-f3b8-4777-b0d2-56bc054366d1'}, 'attachment_id': '858be4aa-e57b-41f7-baa4-4bb0174a0dbd', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:39:23 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:39:23 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'utAhjSFl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:39:23 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:39:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:39:23 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:39:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:39:23 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[eaa77b0a-d9ef-4e33-82e6-63ebd0b4c8be]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:39:23 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2be8645-8dec-4e49-bb36-8dbaf00cc86f]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1] added
Apr 14 04:39:23 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[859829a7-5905-410c-91e8-c898845cec1e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[072ce13e-3f04-4b82-a482-24eb2c1875e3]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:23 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v utAhjSFl {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v utAhjSFl" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[937563be-9895-48a2-98a4-7f1814a1654f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'utAhjSFl'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dce78d63-3904-48ae-bf98-03a22078b14d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[026f3146-3603-426c-9909-42297745a8c7]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c172314c-d994-4338-8d72-c9a6a2f763a5]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1, portal: 172.16.0.220,3260]
Apr 14 04:39:24 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1, portal: 172.16.0.220,3260] successful.
Apr 14 04:39:24 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6139148-eb50-49d2-b234-78d4f2a99b9d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[be7be9b4-94da-4e8d-b483-7f5df91a60b8]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\ntcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)
Apr 14 04:39:24 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.358s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] HCTL ('34', '-', '-', 0) found on session 51 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[64db5062-8ffb-428e-8a2c-92c099860696]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:24 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Searching for a device in session 51 and hctl ['34', '*', '*', 0] yield: None {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Searching for a device in session 51 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'utAhjSFl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.387s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== connect_volume: return (1389ms) {'type': 'block', 'scsi_wwn': '23362643134306261', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23362643134306261', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:39:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-2046991816',display_name='tempest-TestVolumeBootPattern-server-2046991816',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-2046991816',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFZO4o4kQgpEvaFt8iKOk4kfW7YF9jZ6XS2jVS2UAmaxmR76rP/jf7a3pf4xOyQu5V8HIaLR//SNI2rVtnFikRInNPtRWTjIBvrk9d4sXMIgR6FP64tMb+FLBXcO2su56A==',key_name='tempest-TestVolumeBootPattern-1640497739',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-6clzvgqf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:39:19Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=9d93dc7e-d497-4d8f-9ade-c508170f6121,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2fcf791e-22d7-4a96-894c-70900772a5ff", "address": "fa:16:3e:67:49:c3", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2fcf791e-22", "ovs_interfaceid": "2fcf791e-22d7-4a96-894c-70900772a5ff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "2fcf791e-22d7-4a96-894c-70900772a5ff", "address": "fa:16:3e:67:49:c3", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2fcf791e-22", "ovs_interfaceid": "2fcf791e-22d7-4a96-894c-70900772a5ff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:67:49:c3,bridge_name='br-int',has_traffic_filtering=True,id=2fcf791e-22d7-4a96-894c-70900772a5ff,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2fcf791e-22') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG nova.objects.instance [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lazy-loading 'pci_devices' on Instance uuid 9d93dc7e-d497-4d8f-9ade-c508170f6121 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:39:25 user nova-compute[71317]:   <uuid>9d93dc7e-d497-4d8f-9ade-c508170f6121</uuid>
Apr 14 04:39:25 user nova-compute[71317]:   <name>instance-0000002a</name>
Apr 14 04:39:25 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:39:25 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:39:25 user nova-compute[71317]:   <metadata>
Apr 14 04:39:25 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:39:25 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:39:25 user nova-compute[71317]:       <nova:name>tempest-TestVolumeBootPattern-server-2046991816</nova:name>
Apr 14 04:39:25 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:39:23</nova:creationTime>
Apr 14 04:39:25 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:39:25 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:39:25 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:39:25 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:39:25 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:39:25 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:39:25 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:39:25 user nova-compute[71317]:       <nova:owner>
Apr 14 04:39:25 user nova-compute[71317]:         <nova:user uuid="bcf79de7da27495dbfd14c556b5b54c7">tempest-TestVolumeBootPattern-1780160917-project-member</nova:user>
Apr 14 04:39:25 user nova-compute[71317]:         <nova:project uuid="693d95858c624feaa60114aa89082186">tempest-TestVolumeBootPattern-1780160917</nova:project>
Apr 14 04:39:25 user nova-compute[71317]:       </nova:owner>
Apr 14 04:39:25 user nova-compute[71317]:       <nova:ports>
Apr 14 04:39:25 user nova-compute[71317]:         <nova:port uuid="2fcf791e-22d7-4a96-894c-70900772a5ff">
Apr 14 04:39:25 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 14 04:39:25 user nova-compute[71317]:         </nova:port>
Apr 14 04:39:25 user nova-compute[71317]:       </nova:ports>
Apr 14 04:39:25 user nova-compute[71317]:     </nova:instance>
Apr 14 04:39:25 user nova-compute[71317]:   </metadata>
Apr 14 04:39:25 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:39:25 user nova-compute[71317]:     <system>
Apr 14 04:39:25 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:39:25 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:39:25 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:39:25 user nova-compute[71317]:       <entry name="serial">9d93dc7e-d497-4d8f-9ade-c508170f6121</entry>
Apr 14 04:39:25 user nova-compute[71317]:       <entry name="uuid">9d93dc7e-d497-4d8f-9ade-c508170f6121</entry>
Apr 14 04:39:25 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:39:25 user nova-compute[71317]:     </system>
Apr 14 04:39:25 user nova-compute[71317]:   </sysinfo>
Apr 14 04:39:25 user nova-compute[71317]:   <os>
Apr 14 04:39:25 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:39:25 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:39:25 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:39:25 user nova-compute[71317]:   </os>
Apr 14 04:39:25 user nova-compute[71317]:   <features>
Apr 14 04:39:25 user nova-compute[71317]:     <acpi/>
Apr 14 04:39:25 user nova-compute[71317]:     <apic/>
Apr 14 04:39:25 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:39:25 user nova-compute[71317]:   </features>
Apr 14 04:39:25 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:39:25 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:39:25 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:39:25 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:39:25 user nova-compute[71317]:   </clock>
Apr 14 04:39:25 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:39:25 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:39:25 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:39:25 user nova-compute[71317]:   </cpu>
Apr 14 04:39:25 user nova-compute[71317]:   <devices>
Apr 14 04:39:25 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:39:25 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:39:25 user nova-compute[71317]:       <source dev="/dev/sdc"/>
Apr 14 04:39:25 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:39:25 user nova-compute[71317]:       <serial>350da73c-f3b8-4777-b0d2-56bc054366d1</serial>
Apr 14 04:39:25 user nova-compute[71317]:     </disk>
Apr 14 04:39:25 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:39:25 user nova-compute[71317]:       <mac address="fa:16:3e:67:49:c3"/>
Apr 14 04:39:25 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:39:25 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:39:25 user nova-compute[71317]:       <target dev="tap2fcf791e-22"/>
Apr 14 04:39:25 user nova-compute[71317]:     </interface>
Apr 14 04:39:25 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:39:25 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/9d93dc7e-d497-4d8f-9ade-c508170f6121/console.log" append="off"/>
Apr 14 04:39:25 user nova-compute[71317]:     </serial>
Apr 14 04:39:25 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:39:25 user nova-compute[71317]:     <video>
Apr 14 04:39:25 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:39:25 user nova-compute[71317]:     </video>
Apr 14 04:39:25 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:39:25 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:39:25 user nova-compute[71317]:     </rng>
Apr 14 04:39:25 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:39:25 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:39:25 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:39:25 user nova-compute[71317]:     </memballoon>
Apr 14 04:39:25 user nova-compute[71317]:   </devices>
Apr 14 04:39:25 user nova-compute[71317]: </domain>
Apr 14 04:39:25 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:39:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-2046991816',display_name='tempest-TestVolumeBootPattern-server-2046991816',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-2046991816',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFZO4o4kQgpEvaFt8iKOk4kfW7YF9jZ6XS2jVS2UAmaxmR76rP/jf7a3pf4xOyQu5V8HIaLR//SNI2rVtnFikRInNPtRWTjIBvrk9d4sXMIgR6FP64tMb+FLBXcO2su56A==',key_name='tempest-TestVolumeBootPattern-1640497739',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-6clzvgqf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:39:19Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=9d93dc7e-d497-4d8f-9ade-c508170f6121,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2fcf791e-22d7-4a96-894c-70900772a5ff", "address": "fa:16:3e:67:49:c3", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2fcf791e-22", "ovs_interfaceid": "2fcf791e-22d7-4a96-894c-70900772a5ff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "2fcf791e-22d7-4a96-894c-70900772a5ff", "address": "fa:16:3e:67:49:c3", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2fcf791e-22", "ovs_interfaceid": "2fcf791e-22d7-4a96-894c-70900772a5ff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:67:49:c3,bridge_name='br-int',has_traffic_filtering=True,id=2fcf791e-22d7-4a96-894c-70900772a5ff,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2fcf791e-22') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG os_vif [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:67:49:c3,bridge_name='br-int',has_traffic_filtering=True,id=2fcf791e-22d7-4a96-894c-70900772a5ff,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2fcf791e-22') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2fcf791e-22, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2fcf791e-22, col_values=(('external_ids', {'iface-id': '2fcf791e-22d7-4a96-894c-70900772a5ff', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:67:49:c3', 'vm-uuid': '9d93dc7e-d497-4d8f-9ade-c508170f6121'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:25 user nova-compute[71317]: INFO os_vif [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:67:49:c3,bridge_name='br-int',has_traffic_filtering=True,id=2fcf791e-22d7-4a96-894c-70900772a5ff,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2fcf791e-22')
Apr 14 04:39:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:39:25 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] No VIF found with MAC fa:16:3e:67:49:c3, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:39:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-e75f2dd4-fcd2-46ee-af34-5aeced50824e req-261bdde0-e29c-4e2b-a6c2-3f1f451bd194 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Received event network-vif-plugged-2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:39:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e75f2dd4-fcd2-46ee-af34-5aeced50824e req-261bdde0-e29c-4e2b-a6c2-3f1f451bd194 service nova] Acquiring lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e75f2dd4-fcd2-46ee-af34-5aeced50824e req-261bdde0-e29c-4e2b-a6c2-3f1f451bd194 service nova] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e75f2dd4-fcd2-46ee-af34-5aeced50824e req-261bdde0-e29c-4e2b-a6c2-3f1f451bd194 service nova] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-e75f2dd4-fcd2-46ee-af34-5aeced50824e req-261bdde0-e29c-4e2b-a6c2-3f1f451bd194 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] No waiting events found dispatching network-vif-plugged-2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:39:26 user nova-compute[71317]: WARNING nova.compute.manager [req-e75f2dd4-fcd2-46ee-af34-5aeced50824e req-261bdde0-e29c-4e2b-a6c2-3f1f451bd194 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Received unexpected event network-vif-plugged-2fcf791e-22d7-4a96-894c-70900772a5ff for instance with vm_state building and task_state spawning.
Apr 14 04:39:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:27 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447152.8246067, 19a166de-f0c9-43f2-8c88-585e0d5d22cc => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:39:27 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] VM Stopped (Lifecycle Event)
Apr 14 04:39:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-767b9dd5-41eb-4ad4-9dfd-d4c6f0c2c99c None None] [instance: 19a166de-f0c9-43f2-8c88-585e0d5d22cc] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447168.699096, 9d93dc7e-d497-4d8f-9ade-c508170f6121 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:39:28 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] VM Resumed (Lifecycle Event)
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:39:28 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Instance spawned successfully.
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:39:28 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447168.7029204, 9d93dc7e-d497-4d8f-9ade-c508170f6121 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:39:28 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] VM Started (Lifecycle Event)
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:39:28 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:39:28 user nova-compute[71317]: INFO nova.compute.manager [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Took 5.02 seconds to spawn the instance on the hypervisor.
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:39:28 user nova-compute[71317]: INFO nova.compute.manager [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Took 10.33 seconds to build instance.
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-1bbe6a8c-d3de-47d6-ab95-671b6600be91 req-a33c4ad8-e933-4a5e-86e3-b7b14cf6f2d1 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Received event network-vif-plugged-2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1bbe6a8c-d3de-47d6-ab95-671b6600be91 req-a33c4ad8-e933-4a5e-86e3-b7b14cf6f2d1 service nova] Acquiring lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1bbe6a8c-d3de-47d6-ab95-671b6600be91 req-a33c4ad8-e933-4a5e-86e3-b7b14cf6f2d1 service nova] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1bbe6a8c-d3de-47d6-ab95-671b6600be91 req-a33c4ad8-e933-4a5e-86e3-b7b14cf6f2d1 service nova] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-1bbe6a8c-d3de-47d6-ab95-671b6600be91 req-a33c4ad8-e933-4a5e-86e3-b7b14cf6f2d1 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] No waiting events found dispatching network-vif-plugged-2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:39:28 user nova-compute[71317]: WARNING nova.compute.manager [req-1bbe6a8c-d3de-47d6-ab95-671b6600be91 req-a33c4ad8-e933-4a5e-86e3-b7b14cf6f2d1 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Received unexpected event network-vif-plugged-2fcf791e-22d7-4a96-894c-70900772a5ff for instance with vm_state active and task_state None.
Apr 14 04:39:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-60ab6c41-b0c0-4f26-a6c8-b3be552e9445 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.440s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "283ef25b-314a-47c1-8414-9d7c737e5115" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:32 user nova-compute[71317]: INFO nova.compute.manager [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Detaching volume 5615b9ab-7102-4d44-a55e-0e2581c39757
Apr 14 04:39:32 user nova-compute[71317]: INFO nova.virt.block_device [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Attempting to driver detach volume 5615b9ab-7102-4d44-a55e-0e2581c39757 from mountpoint /dev/vdb
Apr 14 04:39:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Attempting to detach device vdb from instance 283ef25b-314a-47c1-8414-9d7c737e5115 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:39:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:39:32 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:39:32 user nova-compute[71317]:   <source dev="/dev/sde"/>
Apr 14 04:39:32 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:39:32 user nova-compute[71317]:   <serial>5615b9ab-7102-4d44-a55e-0e2581c39757</serial>
Apr 14 04:39:32 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:39:32 user nova-compute[71317]: </disk>
Apr 14 04:39:32 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:39:32 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully detached device vdb from instance 283ef25b-314a-47c1-8414-9d7c737e5115 from the persistent domain config.
Apr 14 04:39:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 283ef25b-314a-47c1-8414-9d7c737e5115 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:39:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:39:32 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:39:32 user nova-compute[71317]:   <source dev="/dev/sde"/>
Apr 14 04:39:32 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:39:32 user nova-compute[71317]:   <serial>5615b9ab-7102-4d44-a55e-0e2581c39757</serial>
Apr 14 04:39:32 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:39:32 user nova-compute[71317]: </disk>
Apr 14 04:39:32 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681447173.2923162, 283ef25b-314a-47c1-8414-9d7c737e5115 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 283ef25b-314a-47c1-8414-9d7c737e5115 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:39:33 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully detached device vdb from instance 283ef25b-314a-47c1-8414-9d7c737e5115 from the live domain config.
Apr 14 04:39:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TgIjxlbS', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:39:33 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e42cbc7-bcdb-4b2b-9629-61249fc63737]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:39:33 user nova-compute[71317]: No targets found.
Apr 14 04:39:33 user nova-compute[71317]: iSNS:
Apr 14 04:39:33 user nova-compute[71317]: No targets found.
Apr 14 04:39:33 user nova-compute[71317]: STATIC:
Apr 14 04:39:33 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1
Apr 14 04:39:33 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:39:33 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:39:33 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757
Apr 14 04:39:33 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:39:33 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:39:33 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269
Apr 14 04:39:33 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:39:33 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:39:33 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75
Apr 14 04:39:33 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:39:33 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:39:33 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:39:33 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:39:33 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:39:33 user nova-compute[71317]: FIRMWARE:
Apr 14 04:39:33 user nova-compute[71317]: No targets found.
Apr 14 04:39:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:39:33 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:39:33 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[30ba0084-2a43-45a7-b988-ec95e67dc4fa]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.038s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce96ee37-dc86-4ad9-9959-d19de8524810]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\ntcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:39:33 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:39:33 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:39:33 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:39:33 user nova-compute[71317]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)
Apr 14 04:39:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:39:33 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:39:33 user nova-compute[71317]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 (non-flash)
Apr 14 04:39:33 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:39:33 user nova-compute[71317]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)
Apr 14 04:39:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8490246a70>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757'): ({'sde'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Removing single pathed devices sde {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b4ce29d-4f12-4029-aa6d-f5473b182193]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8df27051-a8fb-48eb-96f1-bbd9dc089f20]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Flushing IO for device /dev/sde {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1500137e-2654-4831-ab86-ba1b7ac37f2d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.048s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3829f65b-a8a5-493e-bf0c-4f8b7220aeab]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] SCSI volumes sde have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.030s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[682c59fe-9e7f-4476-b45b-0b4be83d2685]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --logout" returned: 0 in 0.081s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[766d3c21-3daf-465c-ab12-22a9e6ad9e04]: (4, ('Logging out of session [sid: 49, target: iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757, portal: 172.16.0.220,3260]\nLogout of [sid: 49, target: iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 49, target: iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757, portal: 172.16.0.220,3260]
Apr 14 04:39:33 user nova-compute[71317]: Logout of [sid: 49, target: iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757, portal: 172.16.0.220,3260] successful.
Apr 14 04:39:33 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5615b9ab-7102-4d44-a55e-0e2581c39757 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.032s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd36a0aa-db90-41c5-9aa6-fd8ec26286ac]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.405s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] <== disconnect_volume: return (406ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:39:33 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:39:34 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'flavor' on Instance uuid 283ef25b-314a-47c1-8414-9d7c737e5115 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:39:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5779441c-e5a7-4390-9bce-59c49cafb02a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "283ef25b-314a-47c1-8414-9d7c737e5115" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.207s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:39:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:39:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:39:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:39:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:38 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:39:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:40 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:39:40 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:39:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:39:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:39:40 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:39:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:40 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updating instance_info_cache with network_info: [{"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:39:40 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-3df97db3-fd6c-4be6-af32-7cde9dbcce75" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:39:40 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:39:40 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:41 user nova-compute[71317]: INFO nova.compute.manager [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Terminating instance
Apr 14 04:39:41 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-3544f947-7191-4004-8ca9-3f437b137a9e req-621dfdb1-0162-4de8-992b-bb45877a024b service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received event network-vif-unplugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3544f947-7191-4004-8ca9-3f437b137a9e req-621dfdb1-0162-4de8-992b-bb45877a024b service nova] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3544f947-7191-4004-8ca9-3f437b137a9e req-621dfdb1-0162-4de8-992b-bb45877a024b service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3544f947-7191-4004-8ca9-3f437b137a9e req-621dfdb1-0162-4de8-992b-bb45877a024b service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-3544f947-7191-4004-8ca9-3f437b137a9e req-621dfdb1-0162-4de8-992b-bb45877a024b service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] No waiting events found dispatching network-vif-unplugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG nova.compute.manager [req-3544f947-7191-4004-8ca9-3f437b137a9e req-621dfdb1-0162-4de8-992b-bb45877a024b service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received event network-vif-unplugged-3562cd0d-8f80-4113-a397-b56e4455ef4a for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:41 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Instance destroyed successfully.
Apr 14 04:39:41 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'resources' on Instance uuid 695d931b-1f30-4d57-b6d7-d199d2874e65 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:33:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-372327292',display_name='tempest-ServerActionsTestJSON-server-372327292',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-372327292',id=36,image_ref='',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:35:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-uxrcwktp',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:36:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=695d931b-1f30-4d57-b6d7-d199d2874e65,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "address": "fa:16:3e:f7:65:55", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3562cd0d-8f", "ovs_interfaceid": "3562cd0d-8f80-4113-a397-b56e4455ef4a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG os_vif [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3562cd0d-8f, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:39:41 user nova-compute[71317]: INFO os_vif [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:f7:65:55,bridge_name='br-int',has_traffic_filtering=True,id=3562cd0d-8f80-4113-a397-b56e4455ef4a,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3562cd0d-8f')
Apr 14 04:39:41 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sPgLhMjz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:39:41 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:39:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[13a0001b-8b50-4c66-a28e-df29b1e7f7fb]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:39:41 user nova-compute[71317]: No targets found.
Apr 14 04:39:41 user nova-compute[71317]: iSNS:
Apr 14 04:39:41 user nova-compute[71317]: No targets found.
Apr 14 04:39:41 user nova-compute[71317]: STATIC:
Apr 14 04:39:41 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1
Apr 14 04:39:41 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:39:41 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:39:41 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269
Apr 14 04:39:41 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:39:41 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:39:41 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75
Apr 14 04:39:41 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:39:41 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:39:41 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:39:41 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:39:41 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:39:41 user nova-compute[71317]: FIRMWARE:
Apr 14 04:39:41 user nova-compute[71317]: No targets found.
Apr 14 04:39:41 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:39:41 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:39:41 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[17c9ce4e-3c44-4b87-a340-ee95dab228e3]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[98723beb-cffa-441f-a734-76c9e36d2a1c]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\ntcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:39:41 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:39:41 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:39:41 user nova-compute[71317]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)
Apr 14 04:39:41 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:39:41 user nova-compute[71317]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 (non-flash)
Apr 14 04:39:41 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:39:41 user nova-compute[71317]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)
Apr 14 04:39:41 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84d183c1f0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269'): ({'sdd'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Removing single pathed devices sdd {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7491e63-e0af-4780-b7ee-394d985cb493]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[acd7351f-095e-4c66-a789-fcb7657c50f7]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Flushing IO for device /dev/sdd {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:39:41 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[65c36d63-75cf-4aee-9930-e6b62d006b26]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.046s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bedf871c-5bce-446f-9afa-fc0316a0cc4d]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] SCSI volumes sdd have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-1SCST_BIOf616e73c-e1c3d039cc119196', '/dev/disk/by-id/scsi-26531633364303339', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_f616e73c') {{(pid=81937) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 14 04:39:42 user nova-compute[71317]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-1SCST_BIOf616e73c-e1c3d039cc119196
Apr 14 04:39:42 user nova-compute[71317]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-26531633364303339
Apr 14 04:39:42 user nova-compute[71317]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-SSCST_BIO_Storage_f616e73c
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4dd68cca-34ab-4a0d-8213-3fe968969ff9]: (4, None) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5ce3a93-c961-4693-8816-703fc5875182]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --logout" returned: 0 in 0.068s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[71033549-296d-4726-9e7a-f86f8281db5e]: (4, ('Logging out of session [sid: 47, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260]\nLogout of [sid: 47, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 47, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260]
Apr 14 04:39:42 user nova-compute[71317]: Logout of [sid: 47, target: iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269, portal: 172.16.0.220,3260] successful.
Apr 14 04:39:42 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:992a3f86-d313-4b1e-9491-3008be037269 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ff96b64-8913-419b-8fa2-9e9d9cc8e00e]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.330s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] <== disconnect_volume: return (331ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:39:42 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Deleting instance files /opt/stack/data/nova/instances/695d931b-1f30-4d57-b6d7-d199d2874e65_del
Apr 14 04:39:42 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Deletion of /opt/stack/data/nova/instances/695d931b-1f30-4d57-b6d7-d199d2874e65_del complete
Apr 14 04:39:42 user nova-compute[71317]: INFO nova.compute.manager [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Took 1.21 seconds to destroy the instance on the hypervisor.
Apr 14 04:39:42 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:39:42 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:39:43 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:39:43 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Took 0.80 seconds to deallocate network for instance.
Apr 14 04:39:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-1f31f810-aa52-4daa-a17d-14b883426b96 req-83100610-7055-4009-9345-495a814e943c service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received event network-vif-deleted-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:39:43 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:39:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-07c5c969-4c88-4ef4-9e02-6e4fc4333078 req-590f6175-8699-4055-b2fc-666f583da460 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received event network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:39:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-07c5c969-4c88-4ef4-9e02-6e4fc4333078 req-590f6175-8699-4055-b2fc-666f583da460 service nova] Acquiring lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-07c5c969-4c88-4ef4-9e02-6e4fc4333078 req-590f6175-8699-4055-b2fc-666f583da460 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-07c5c969-4c88-4ef4-9e02-6e4fc4333078 req-590f6175-8699-4055-b2fc-666f583da460 service nova] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:43 user nova-compute[71317]: DEBUG nova.compute.manager [req-07c5c969-4c88-4ef4-9e02-6e4fc4333078 req-590f6175-8699-4055-b2fc-666f583da460 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] No waiting events found dispatching network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:39:43 user nova-compute[71317]: WARNING nova.compute.manager [req-07c5c969-4c88-4ef4-9e02-6e4fc4333078 req-590f6175-8699-4055-b2fc-666f583da460 service nova] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Received unexpected event network-vif-plugged-3562cd0d-8f80-4113-a397-b56e4455ef4a for instance with vm_state active and task_state deleting.
Apr 14 04:39:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:43 user nova-compute[71317]: INFO nova.compute.manager [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Took 0.77 seconds to detach 1 volumes for instance.
Apr 14 04:39:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Deleting volume: 992a3f86-d313-4b1e-9491-3008be037269 {{(pid=71317) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 14 04:39:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:44 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:39:44 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:39:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.240s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:44 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Deleted allocations for instance 695d931b-1f30-4d57-b6d7-d199d2874e65
Apr 14 04:39:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d6bff91f-e726-4850-ae52-d41bdcb3e0b1 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "695d931b-1f30-4d57-b6d7-d199d2874e65" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.270s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:46 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:39:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:39:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:39:49 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:39:49 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:39:49 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8231MB free_disk=26.249141693115234GB free_vcpus=6 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 283ef25b-314a-47c1-8414-9d7c737e5115 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 49c7ae26-8d9e-45ac-a189-3ee84e015b27 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 9d93dc7e-d497-4d8f-9ade-c508170f6121 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 6 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:39:49 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=1280MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=6 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:39:50 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:39:50 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:39:50 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:39:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.479s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:39:51 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:39:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:39:51 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:39:56 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447181.8010879, 695d931b-1f30-4d57-b6d7-d199d2874e65 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:39:56 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] VM Stopped (Lifecycle Event)
Apr 14 04:39:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-f2ef7b6e-d493-45c6-b297-f2519ffde00f None None] [instance: 695d931b-1f30-4d57-b6d7-d199d2874e65] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:39:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:39:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:02 user nova-compute[71317]: INFO nova.compute.manager [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Terminating instance
Apr 14 04:40:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:40:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-7b077c91-7e6d-4f59-a2a1-0ef14d2d20bc req-cb03e486-1e9d-4057-8b44-bb8e6a0ab53e service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-unplugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7b077c91-7e6d-4f59-a2a1-0ef14d2d20bc req-cb03e486-1e9d-4057-8b44-bb8e6a0ab53e service nova] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7b077c91-7e6d-4f59-a2a1-0ef14d2d20bc req-cb03e486-1e9d-4057-8b44-bb8e6a0ab53e service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7b077c91-7e6d-4f59-a2a1-0ef14d2d20bc req-cb03e486-1e9d-4057-8b44-bb8e6a0ab53e service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-7b077c91-7e6d-4f59-a2a1-0ef14d2d20bc req-cb03e486-1e9d-4057-8b44-bb8e6a0ab53e service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] No waiting events found dispatching network-vif-unplugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:40:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-7b077c91-7e6d-4f59-a2a1-0ef14d2d20bc req-cb03e486-1e9d-4057-8b44-bb8e6a0ab53e service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-unplugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Instance destroyed successfully.
Apr 14 04:40:03 user nova-compute[71317]: DEBUG nova.objects.instance [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'resources' on Instance uuid 3df97db3-fd6c-4be6-af32-7cde9dbcce75 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:24:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1117599705',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1117599705',id=24,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:26:16Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-5xp7dr1d',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:28:03Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=3df97db3-fd6c-4be6-af32-7cde9dbcce75,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "address": "fa:16:3e:f6:fa:7c", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6baf0637-1e", "ovs_interfaceid": "6baf0637-1ee9-4ccb-93a3-264041fb73f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f6:fa:7c,bridge_name='br-int',has_traffic_filtering=True,id=6baf0637-1ee9-4ccb-93a3-264041fb73f3,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6baf0637-1e') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_vif [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:f6:fa:7c,bridge_name='br-int',has_traffic_filtering=True,id=6baf0637-1ee9-4ccb-93a3-264041fb73f3,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6baf0637-1e') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap6baf0637-1e, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:40:03 user nova-compute[71317]: INFO os_vif [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:f6:fa:7c,bridge_name='br-int',has_traffic_filtering=True,id=6baf0637-1ee9-4ccb-93a3-264041fb73f3,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6baf0637-1e')
Apr 14 04:40:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YCyASkTZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:40:03 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[01ade408-35fa-48e8-ba09-a0763deed065]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:40:03 user nova-compute[71317]: No targets found.
Apr 14 04:40:03 user nova-compute[71317]: iSNS:
Apr 14 04:40:03 user nova-compute[71317]: No targets found.
Apr 14 04:40:03 user nova-compute[71317]: STATIC:
Apr 14 04:40:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1
Apr 14 04:40:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:40:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:40:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75
Apr 14 04:40:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:40:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:40:03 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866
Apr 14 04:40:03 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:40:03 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:40:03 user nova-compute[71317]: FIRMWARE:
Apr 14 04:40:03 user nova-compute[71317]: No targets found.
Apr 14 04:40:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:40:03 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:40:03 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b34558c-e8a7-486b-9789-7985e92908c3]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[61fd2ee4-7de4-4f9a-8c4a-ac571c381490]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\ntcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:40:03 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:40:03 user nova-compute[71317]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)
Apr 14 04:40:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 (non-flash)
Apr 14 04:40:03 user nova-compute[71317]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:40:03 user nova-compute[71317]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)
Apr 14 04:40:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f849025e5f0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866'): ({'sdb'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Removing single pathed devices sdb {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c87043b-c888-49c2-868d-489f08be6997]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[48681a30-6b5f-4a01-95ae-8f213888062a]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Flushing IO for device /dev/sdb {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[de40cacf-45cf-4139-aaf5-492138c745d5]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.043s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4cd0097d-87b8-4207-b34d-a6a2d59ea80d]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9cd2d9b7-06ed-4568-9cb0-186006927d1a]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[38521a16-b19b-4b43-aa99-985f53b399ee]: (4, ('Logging out of session [sid: 23, target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866, portal: 172.16.0.220,3260]\nLogout of [sid: 23, target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 23, target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866, portal: 172.16.0.220,3260]
Apr 14 04:40:03 user nova-compute[71317]: Logout of [sid: 23, target: iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866, portal: 172.16.0.220,3260] successful.
Apr 14 04:40:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e31b954-88d8-448b-bbb7-f19007a51866 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[894fe084-5fff-4652-a826-22670d433bba]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.303s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] <== disconnect_volume: return (304ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:40:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Deleting instance files /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75_del
Apr 14 04:40:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Deletion of /opt/stack/data/nova/instances/3df97db3-fd6c-4be6-af32-7cde9dbcce75_del complete
Apr 14 04:40:03 user nova-compute[71317]: INFO nova.compute.manager [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Took 1.14 seconds to destroy the instance on the hypervisor.
Apr 14 04:40:03 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:40:03 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:40:04 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:40:04 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Took 0.47 seconds to deallocate network for instance.
Apr 14 04:40:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-2bad1a3b-8a26-45b9-ad7a-508b0f153292 req-702338aa-35ee-4b21-b65c-8c9456e3b4b9 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-deleted-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:05 user nova-compute[71317]: DEBUG nova.compute.manager [req-2999db5e-7e95-4b1f-810a-79ec6400f83e req-aec0dfa6-37c2-4528-95b7-02c4d03e14e4 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2999db5e-7e95-4b1f-810a-79ec6400f83e req-aec0dfa6-37c2-4528-95b7-02c4d03e14e4 service nova] Acquiring lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2999db5e-7e95-4b1f-810a-79ec6400f83e req-aec0dfa6-37c2-4528-95b7-02c4d03e14e4 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2999db5e-7e95-4b1f-810a-79ec6400f83e req-aec0dfa6-37c2-4528-95b7-02c4d03e14e4 service nova] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:05 user nova-compute[71317]: DEBUG nova.compute.manager [req-2999db5e-7e95-4b1f-810a-79ec6400f83e req-aec0dfa6-37c2-4528-95b7-02c4d03e14e4 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] No waiting events found dispatching network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:40:05 user nova-compute[71317]: WARNING nova.compute.manager [req-2999db5e-7e95-4b1f-810a-79ec6400f83e req-aec0dfa6-37c2-4528-95b7-02c4d03e14e4 service nova] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Received unexpected event network-vif-plugged-6baf0637-1ee9-4ccb-93a3-264041fb73f3 for instance with vm_state active and task_state deleting.
Apr 14 04:40:05 user nova-compute[71317]: INFO nova.compute.manager [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Took 0.79 seconds to detach 1 volumes for instance.
Apr 14 04:40:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:05 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:40:05 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:40:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.209s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:05 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Deleted allocations for instance 3df97db3-fd6c-4be6-af32-7cde9dbcce75
Apr 14 04:40:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-ee648a24-65c5-441a-a367-c24fa9284c53 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "3df97db3-fd6c-4be6-af32-7cde9dbcce75" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.768s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:18 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447203.4026234, 3df97db3-fd6c-4be6-af32-7cde9dbcce75 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:40:18 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] VM Stopped (Lifecycle Event)
Apr 14 04:40:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-bcb92d11-e2d5-476c-905c-6cab4e8e0dde None None] [instance: 3df97db3-fd6c-4be6-af32-7cde9dbcce75] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:40:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "283ef25b-314a-47c1-8414-9d7c737e5115" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:22 user nova-compute[71317]: INFO nova.compute.manager [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Terminating instance
Apr 14 04:40:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-833a5a82-27e9-46ab-b569-5ee09d3788b7 req-cf5b90e6-7f70-456a-8daf-5ed899a9ac5f service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received event network-vif-unplugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-833a5a82-27e9-46ab-b569-5ee09d3788b7 req-cf5b90e6-7f70-456a-8daf-5ed899a9ac5f service nova] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-833a5a82-27e9-46ab-b569-5ee09d3788b7 req-cf5b90e6-7f70-456a-8daf-5ed899a9ac5f service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-833a5a82-27e9-46ab-b569-5ee09d3788b7 req-cf5b90e6-7f70-456a-8daf-5ed899a9ac5f service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-833a5a82-27e9-46ab-b569-5ee09d3788b7 req-cf5b90e6-7f70-456a-8daf-5ed899a9ac5f service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] No waiting events found dispatching network-vif-unplugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-833a5a82-27e9-46ab-b569-5ee09d3788b7 req-cf5b90e6-7f70-456a-8daf-5ed899a9ac5f service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received event network-vif-unplugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:23 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Instance destroyed successfully.
Apr 14 04:40:23 user nova-compute[71317]: DEBUG nova.objects.instance [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'resources' on Instance uuid 283ef25b-314a-47c1-8414-9d7c737e5115 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:34:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-930330581',display_name='tempest-AttachVolumeNegativeTest-server-930330581',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-930330581',id=39,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLkMrxFUn3dpVh/tPu0RpQlCyFBaJgCQyQYEwDhdWuIFHQhRjgZJy1F9LImzGW0YKeJb/lrhOq2b/KwQ6C50SgVO3YvlPJwAtkiYLlWF4eUkqOgYRYwPGHjor09Iuea8tQ==',key_name='tempest-keypair-499854002',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:34:51Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='601e2806907a48d3995cc90b54da3c8c',ramdisk_id='',reservation_id='r-ceukq3d3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-1197484356',owner_user_name='tempest-AttachVolumeNegativeTest-1197484356-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:34:52Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7cab2461ab0d4eb08b62feaf6396bc87',uuid=283ef25b-314a-47c1-8414-9d7c737e5115,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "address": "fa:16:3e:6a:02:89", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.172", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4734e3dd-22", "ovs_interfaceid": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converting VIF {"id": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "address": "fa:16:3e:6a:02:89", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.172", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4734e3dd-22", "ovs_interfaceid": "4734e3dd-22a1-4d25-b2cd-e19ef4f66f83", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:6a:02:89,bridge_name='br-int',has_traffic_filtering=True,id=4734e3dd-22a1-4d25-b2cd-e19ef4f66f83,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4734e3dd-22') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG os_vif [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:6a:02:89,bridge_name='br-int',has_traffic_filtering=True,id=4734e3dd-22a1-4d25-b2cd-e19ef4f66f83,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4734e3dd-22') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap4734e3dd-22, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:40:23 user nova-compute[71317]: INFO os_vif [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:6a:02:89,bridge_name='br-int',has_traffic_filtering=True,id=4734e3dd-22a1-4d25-b2cd-e19ef4f66f83,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4734e3dd-22')
Apr 14 04:40:23 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Deleting instance files /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115_del
Apr 14 04:40:23 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Deletion of /opt/stack/data/nova/instances/283ef25b-314a-47c1-8414-9d7c737e5115_del complete
Apr 14 04:40:23 user nova-compute[71317]: INFO nova.compute.manager [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Took 0.88 seconds to destroy the instance on the hypervisor.
Apr 14 04:40:23 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:40:23 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:40:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:24 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:40:24 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Took 0.88 seconds to deallocate network for instance.
Apr 14 04:40:24 user nova-compute[71317]: DEBUG nova.compute.manager [req-3f77e5c5-c67c-4e22-b949-d15739c6dd07 req-10bb0de1-f74b-4fc8-b565-c069559351e9 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received event network-vif-deleted-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:24 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:40:24 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.210s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:25 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Deleted allocations for instance 283ef25b-314a-47c1-8414-9d7c737e5115
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-e83b3325-f30f-4402-a4ee-490fe8292a22 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "283ef25b-314a-47c1-8414-9d7c737e5115" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.158s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received event network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] No waiting events found dispatching network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:40:25 user nova-compute[71317]: WARNING nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received unexpected event network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 for instance with vm_state deleted and task_state None.
Apr 14 04:40:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received event network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] No waiting events found dispatching network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:40:25 user nova-compute[71317]: WARNING nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received unexpected event network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 for instance with vm_state deleted and task_state None.
Apr 14 04:40:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received event network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] No waiting events found dispatching network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:40:25 user nova-compute[71317]: WARNING nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received unexpected event network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 for instance with vm_state deleted and task_state None.
Apr 14 04:40:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received event network-vif-unplugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] No waiting events found dispatching network-vif-unplugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:40:25 user nova-compute[71317]: WARNING nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received unexpected event network-vif-unplugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 for instance with vm_state deleted and task_state None.
Apr 14 04:40:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received event network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Acquiring lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] Lock "283ef25b-314a-47c1-8414-9d7c737e5115-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] No waiting events found dispatching network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:40:25 user nova-compute[71317]: WARNING nova.compute.manager [req-81a94549-71d5-41c2-b2e6-fbc423429e28 req-91e8c73f-e209-4412-a20f-bc22083e9d69 service nova] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Received unexpected event network-vif-plugged-4734e3dd-22a1-4d25-b2cd-e19ef4f66f83 for instance with vm_state deleted and task_state None.
Apr 14 04:40:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:31 user nova-compute[71317]: INFO nova.compute.manager [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Terminating instance
Apr 14 04:40:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG nova.compute.manager [req-2cdf4033-7f62-4e30-a304-006fefa2ec80 req-5ca3a284-efe6-48f0-a9d1-7e7deef2c221 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Received event network-vif-unplugged-bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2cdf4033-7f62-4e30-a304-006fefa2ec80 req-5ca3a284-efe6-48f0-a9d1-7e7deef2c221 service nova] Acquiring lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2cdf4033-7f62-4e30-a304-006fefa2ec80 req-5ca3a284-efe6-48f0-a9d1-7e7deef2c221 service nova] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2cdf4033-7f62-4e30-a304-006fefa2ec80 req-5ca3a284-efe6-48f0-a9d1-7e7deef2c221 service nova] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG nova.compute.manager [req-2cdf4033-7f62-4e30-a304-006fefa2ec80 req-5ca3a284-efe6-48f0-a9d1-7e7deef2c221 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] No waiting events found dispatching network-vif-unplugged-bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:40:31 user nova-compute[71317]: DEBUG nova.compute.manager [req-2cdf4033-7f62-4e30-a304-006fefa2ec80 req-5ca3a284-efe6-48f0-a9d1-7e7deef2c221 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Received event network-vif-unplugged-bc83f476-5176-4301-8272-80034f4a0daf for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:40:32 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Instance destroyed successfully.
Apr 14 04:40:32 user nova-compute[71317]: DEBUG nova.objects.instance [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lazy-loading 'resources' on Instance uuid 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:31:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-496557296',display_name='tempest-ServerActionsTestJSON-server-496557296',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-496557296',id=34,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhQVAon6iUmWgfiTnbVVO2wSLx2ACi4ZUupr/3b/O255im7X/6hIJW49PlgfIgMrBQ+6w5QNr5f28LOri03H7nhpJeLG+aJessXLUSHzOfJ6DGDJhDGy+XFKNWimHOKdQ==',key_name='tempest-keypair-964114001',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:31:16Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='8c14c332dfcc44b49ddf3cb482dd0d4c',ramdisk_id='',reservation_id='r-olssq35z',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1048786641',owner_user_name='tempest-ServerActionsTestJSON-1048786641-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:31:17Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e35b3ae371cf4dd4b21e2999e58cd2f4',uuid=123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converting VIF {"id": "bc83f476-5176-4301-8272-80034f4a0daf", "address": "fa:16:3e:69:b3:83", "network": {"id": "2fe4c44a-0ec3-4df5-a079-ae920ed93fe7", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1431385934-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c14c332dfcc44b49ddf3cb482dd0d4c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbc83f476-51", "ovs_interfaceid": "bc83f476-5176-4301-8272-80034f4a0daf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:69:b3:83,bridge_name='br-int',has_traffic_filtering=True,id=bc83f476-5176-4301-8272-80034f4a0daf,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbc83f476-51') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG os_vif [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:69:b3:83,bridge_name='br-int',has_traffic_filtering=True,id=bc83f476-5176-4301-8272-80034f4a0daf,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbc83f476-51') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapbc83f476-51, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:40:32 user nova-compute[71317]: INFO os_vif [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:69:b3:83,bridge_name='br-int',has_traffic_filtering=True,id=bc83f476-5176-4301-8272-80034f4a0daf,network=Network(2fe4c44a-0ec3-4df5-a079-ae920ed93fe7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbc83f476-51')
Apr 14 04:40:32 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Deleting instance files /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c_del
Apr 14 04:40:32 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Deletion of /opt/stack/data/nova/instances/123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c_del complete
Apr 14 04:40:32 user nova-compute[71317]: INFO nova.compute.manager [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Took 0.66 seconds to destroy the instance on the hypervisor.
Apr 14 04:40:32 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:40:32 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Took 0.46 seconds to deallocate network for instance.
Apr 14 04:40:32 user nova-compute[71317]: DEBUG nova.compute.manager [req-064982e0-62a8-43b5-810a-c8d5d9c8a5a8 req-07d45e47-6bed-430d-a3cb-09c3bf6069e2 service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Received event network-vif-deleted-bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:40:32 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:40:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.271s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:33 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Deleted allocations for instance 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c
Apr 14 04:40:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4ff0ecda-4909-4c29-b60a-64ac75e7e3f4 tempest-ServerActionsTestJSON-1048786641 tempest-ServerActionsTestJSON-1048786641-project-member] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.642s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:33 user nova-compute[71317]: DEBUG nova.compute.manager [req-059c24e5-efeb-46cb-a6fb-f66fd9e724a0 req-c22eb9db-f5a2-489d-a70e-2b0bb8360b2d service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Received event network-vif-plugged-bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-059c24e5-efeb-46cb-a6fb-f66fd9e724a0 req-c22eb9db-f5a2-489d-a70e-2b0bb8360b2d service nova] Acquiring lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-059c24e5-efeb-46cb-a6fb-f66fd9e724a0 req-c22eb9db-f5a2-489d-a70e-2b0bb8360b2d service nova] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:33 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-059c24e5-efeb-46cb-a6fb-f66fd9e724a0 req-c22eb9db-f5a2-489d-a70e-2b0bb8360b2d service nova] Lock "123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:33 user nova-compute[71317]: DEBUG nova.compute.manager [req-059c24e5-efeb-46cb-a6fb-f66fd9e724a0 req-c22eb9db-f5a2-489d-a70e-2b0bb8360b2d service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] No waiting events found dispatching network-vif-plugged-bc83f476-5176-4301-8272-80034f4a0daf {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:40:33 user nova-compute[71317]: WARNING nova.compute.manager [req-059c24e5-efeb-46cb-a6fb-f66fd9e724a0 req-c22eb9db-f5a2-489d-a70e-2b0bb8360b2d service nova] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Received unexpected event network-vif-plugged-bc83f476-5176-4301-8272-80034f4a0daf for instance with vm_state deleted and task_state None.
Apr 14 04:40:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:38 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:40:38 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447223.720147, 283ef25b-314a-47c1-8414-9d7c737e5115 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:40:38 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] VM Stopped (Lifecycle Event)
Apr 14 04:40:38 user nova-compute[71317]: DEBUG nova.compute.manager [None req-9f9feaf9-f3a6-4e04-b842-d3b860cbd26b None None] [instance: 283ef25b-314a-47c1-8414-9d7c737e5115] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:40:42 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:40:42 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:40:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:40:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:40:42 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:40:42 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:40:43 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updating instance_info_cache with network_info: [{"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.171", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:40:43 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:40:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:40:43 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:40:43 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:40:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:47 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447232.1545784, 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:40:47 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] VM Stopped (Lifecycle Event)
Apr 14 04:40:47 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:40:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-7466a853-3d66-4b22-bc2b-e47e8a4bb9d0 None None] [instance: 123cbab6-0dcc-4f1c-a6c0-89eed5b41b7c] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:40:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:40:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:40:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:49 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:40:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:40:49 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:40:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:40:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:40:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:40:49 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:40:50 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:40:50 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:40:50 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8463MB free_disk=26.287311553955078GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:40:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:50 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:40:50 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 49c7ae26-8d9e-45ac-a189-3ee84e015b27 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:40:50 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 9d93dc7e-d497-4d8f-9ade-c508170f6121 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:40:50 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:40:50 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:40:50 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:40:50 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:40:50 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:40:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.257s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:51 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:40:51 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:40:51 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:40:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:53 user nova-compute[71317]: INFO nova.compute.manager [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Terminating instance
Apr 14 04:40:53 user nova-compute[71317]: DEBUG nova.compute.manager [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-89bc58a9-8141-49c8-ab22-0088bb9f8585 req-20203649-7559-4e44-8192-5eea584c7121 service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Received event network-vif-unplugged-01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-89bc58a9-8141-49c8-ab22-0088bb9f8585 req-20203649-7559-4e44-8192-5eea584c7121 service nova] Acquiring lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-89bc58a9-8141-49c8-ab22-0088bb9f8585 req-20203649-7559-4e44-8192-5eea584c7121 service nova] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-89bc58a9-8141-49c8-ab22-0088bb9f8585 req-20203649-7559-4e44-8192-5eea584c7121 service nova] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-89bc58a9-8141-49c8-ab22-0088bb9f8585 req-20203649-7559-4e44-8192-5eea584c7121 service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] No waiting events found dispatching network-vif-unplugged-01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:40:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-89bc58a9-8141-49c8-ab22-0088bb9f8585 req-20203649-7559-4e44-8192-5eea584c7121 service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Received event network-vif-unplugged-01c99489-4391-41b0-81d9-212d7e76b28c for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:40:53 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Instance destroyed successfully.
Apr 14 04:40:53 user nova-compute[71317]: DEBUG nova.objects.instance [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lazy-loading 'resources' on Instance uuid 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:21:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1631037722',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1631037722',id=18,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:21:22Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d5995b934b0546c0983387e46e3b3df4',ramdisk_id='',reservation_id='r-8nem9px3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-247821842',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:23:13Z,user_data=None,user_id='6ec7103e06e643a6b9b2c908ba8d7b06',uuid=226b0e81-b6a2-4094-b0f1-3fb0d3b94074,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converting VIF {"id": "01c99489-4391-41b0-81d9-212d7e76b28c", "address": "fa:16:3e:59:f9:6e", "network": {"id": "7457ae38-8cd9-4172-aa34-25b764436aee", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1512364937-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d5995b934b0546c0983387e46e3b3df4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap01c99489-43", "ovs_interfaceid": "01c99489-4391-41b0-81d9-212d7e76b28c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:59:f9:6e,bridge_name='br-int',has_traffic_filtering=True,id=01c99489-4391-41b0-81d9-212d7e76b28c,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap01c99489-43') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG os_vif [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:59:f9:6e,bridge_name='br-int',has_traffic_filtering=True,id=01c99489-4391-41b0-81d9-212d7e76b28c,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap01c99489-43') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap01c99489-43, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:40:54 user nova-compute[71317]: INFO os_vif [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:59:f9:6e,bridge_name='br-int',has_traffic_filtering=True,id=01c99489-4391-41b0-81d9-212d7e76b28c,network=Network(7457ae38-8cd9-4172-aa34-25b764436aee),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap01c99489-43')
Apr 14 04:40:54 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Deleting instance files /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074_del
Apr 14 04:40:54 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Deletion of /opt/stack/data/nova/instances/226b0e81-b6a2-4094-b0f1-3fb0d3b94074_del complete
Apr 14 04:40:54 user nova-compute[71317]: INFO nova.compute.manager [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Took 0.69 seconds to destroy the instance on the hypervisor.
Apr 14 04:40:54 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:40:54 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Took 0.50 seconds to deallocate network for instance.
Apr 14 04:40:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-9b434088-8dda-40e6-9448-da9022283448 req-d7777d6a-52f8-493d-9d11-a95fad5fac34 service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Received event network-vif-deleted-01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:40:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.151s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:54 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Deleted allocations for instance 226b0e81-b6a2-4094-b0f1-3fb0d3b94074
Apr 14 04:40:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-5833dd11-8967-47e4-8fb9-a7c893ed17d9 tempest-ServerBootFromVolumeStableRescueTest-247821842 tempest-ServerBootFromVolumeStableRescueTest-247821842-project-member] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.520s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-c139f918-ddd2-48b1-a5d4-af4aade02076 req-c7492b30-45d2-4f55-a3da-0ddcf07c3b69 service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Received event network-vif-plugged-01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:40:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c139f918-ddd2-48b1-a5d4-af4aade02076 req-c7492b30-45d2-4f55-a3da-0ddcf07c3b69 service nova] Acquiring lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:40:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c139f918-ddd2-48b1-a5d4-af4aade02076 req-c7492b30-45d2-4f55-a3da-0ddcf07c3b69 service nova] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:40:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c139f918-ddd2-48b1-a5d4-af4aade02076 req-c7492b30-45d2-4f55-a3da-0ddcf07c3b69 service nova] Lock "226b0e81-b6a2-4094-b0f1-3fb0d3b94074-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:40:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-c139f918-ddd2-48b1-a5d4-af4aade02076 req-c7492b30-45d2-4f55-a3da-0ddcf07c3b69 service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] No waiting events found dispatching network-vif-plugged-01c99489-4391-41b0-81d9-212d7e76b28c {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:40:55 user nova-compute[71317]: WARNING nova.compute.manager [req-c139f918-ddd2-48b1-a5d4-af4aade02076 req-c7492b30-45d2-4f55-a3da-0ddcf07c3b69 service nova] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Received unexpected event network-vif-plugged-01c99489-4391-41b0-81d9-212d7e76b28c for instance with vm_state deleted and task_state None.
Apr 14 04:40:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:41:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:41:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:41:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:41:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:08 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447253.9772296, 226b0e81-b6a2-4094-b0f1-3fb0d3b94074 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:41:08 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] VM Stopped (Lifecycle Event)
Apr 14 04:41:09 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a083ef29-f968-4efa-8c5e-6fd8e94979c4 None None] [instance: 226b0e81-b6a2-4094-b0f1-3fb0d3b94074] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:41:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-fdd78410-e9eb-46c4-bea3-dc4c1e2496cc req-b7479e5c-c3f6-4b57-b824-4d34d8bfffb5 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Received event network-changed-2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-fdd78410-e9eb-46c4-bea3-dc4c1e2496cc req-b7479e5c-c3f6-4b57-b824-4d34d8bfffb5 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Refreshing instance network info cache due to event network-changed-2fcf791e-22d7-4a96-894c-70900772a5ff. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-fdd78410-e9eb-46c4-bea3-dc4c1e2496cc req-b7479e5c-c3f6-4b57-b824-4d34d8bfffb5 service nova] Acquiring lock "refresh_cache-9d93dc7e-d497-4d8f-9ade-c508170f6121" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-fdd78410-e9eb-46c4-bea3-dc4c1e2496cc req-b7479e5c-c3f6-4b57-b824-4d34d8bfffb5 service nova] Acquired lock "refresh_cache-9d93dc7e-d497-4d8f-9ade-c508170f6121" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG nova.network.neutron [req-fdd78410-e9eb-46c4-bea3-dc4c1e2496cc req-b7479e5c-c3f6-4b57-b824-4d34d8bfffb5 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Refreshing network info cache for port 2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG nova.network.neutron [req-fdd78410-e9eb-46c4-bea3-dc4c1e2496cc req-b7479e5c-c3f6-4b57-b824-4d34d8bfffb5 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Updated VIF entry in instance network info cache for port 2fcf791e-22d7-4a96-894c-70900772a5ff. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG nova.network.neutron [req-fdd78410-e9eb-46c4-bea3-dc4c1e2496cc req-b7479e5c-c3f6-4b57-b824-4d34d8bfffb5 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Updating instance_info_cache with network_info: [{"id": "2fcf791e-22d7-4a96-894c-70900772a5ff", "address": "fa:16:3e:67:49:c3", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.181", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2fcf791e-22", "ovs_interfaceid": "2fcf791e-22d7-4a96-894c-70900772a5ff", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-fdd78410-e9eb-46c4-bea3-dc4c1e2496cc req-b7479e5c-c3f6-4b57-b824-4d34d8bfffb5 service nova] Releasing lock "refresh_cache-9d93dc7e-d497-4d8f-9ade-c508170f6121" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "9d93dc7e-d497-4d8f-9ade-c508170f6121" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.005s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:10 user nova-compute[71317]: INFO nova.compute.manager [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Terminating instance
Apr 14 04:41:10 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-2346b874-4e93-4387-aa29-25ceb406b250 req-3f239bc3-4e28-4e3b-988f-cf3add4df1a6 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Received event network-vif-unplugged-2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2346b874-4e93-4387-aa29-25ceb406b250 req-3f239bc3-4e28-4e3b-988f-cf3add4df1a6 service nova] Acquiring lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2346b874-4e93-4387-aa29-25ceb406b250 req-3f239bc3-4e28-4e3b-988f-cf3add4df1a6 service nova] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2346b874-4e93-4387-aa29-25ceb406b250 req-3f239bc3-4e28-4e3b-988f-cf3add4df1a6 service nova] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-2346b874-4e93-4387-aa29-25ceb406b250 req-3f239bc3-4e28-4e3b-988f-cf3add4df1a6 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] No waiting events found dispatching network-vif-unplugged-2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG nova.compute.manager [req-2346b874-4e93-4387-aa29-25ceb406b250 req-3f239bc3-4e28-4e3b-988f-cf3add4df1a6 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Received event network-vif-unplugged-2fcf791e-22d7-4a96-894c-70900772a5ff for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:11 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Instance destroyed successfully.
Apr 14 04:41:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lazy-loading 'resources' on Instance uuid 9d93dc7e-d497-4d8f-9ade-c508170f6121 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:39:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-2046991816',display_name='tempest-TestVolumeBootPattern-server-2046991816',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-2046991816',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFZO4o4kQgpEvaFt8iKOk4kfW7YF9jZ6XS2jVS2UAmaxmR76rP/jf7a3pf4xOyQu5V8HIaLR//SNI2rVtnFikRInNPtRWTjIBvrk9d4sXMIgR6FP64tMb+FLBXcO2su56A==',key_name='tempest-TestVolumeBootPattern-1640497739',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:39:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-6clzvgqf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:39:29Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=9d93dc7e-d497-4d8f-9ade-c508170f6121,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2fcf791e-22d7-4a96-894c-70900772a5ff", "address": "fa:16:3e:67:49:c3", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.181", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2fcf791e-22", "ovs_interfaceid": "2fcf791e-22d7-4a96-894c-70900772a5ff", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "2fcf791e-22d7-4a96-894c-70900772a5ff", "address": "fa:16:3e:67:49:c3", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.181", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2fcf791e-22", "ovs_interfaceid": "2fcf791e-22d7-4a96-894c-70900772a5ff", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:67:49:c3,bridge_name='br-int',has_traffic_filtering=True,id=2fcf791e-22d7-4a96-894c-70900772a5ff,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2fcf791e-22') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_vif [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:67:49:c3,bridge_name='br-int',has_traffic_filtering=True,id=2fcf791e-22d7-4a96-894c-70900772a5ff,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2fcf791e-22') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2fcf791e-22, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:41:11 user nova-compute[71317]: INFO os_vif [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:67:49:c3,bridge_name='br-int',has_traffic_filtering=True,id=2fcf791e-22d7-4a96-894c-70900772a5ff,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2fcf791e-22')
Apr 14 04:41:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'utAhjSFl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:41:11 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[38507a6d-5405-4497-8712-257fec88a9d0]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:41:11 user nova-compute[71317]: No targets found.
Apr 14 04:41:11 user nova-compute[71317]: iSNS:
Apr 14 04:41:11 user nova-compute[71317]: No targets found.
Apr 14 04:41:11 user nova-compute[71317]: STATIC:
Apr 14 04:41:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1
Apr 14 04:41:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:41:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:41:11 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75
Apr 14 04:41:11 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:41:11 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:41:11 user nova-compute[71317]: FIRMWARE:
Apr 14 04:41:11 user nova-compute[71317]: No targets found.
Apr 14 04:41:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:41:11 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:41:11 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2fd03ae-ac9c-447b-8383-3179722845cc]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[19f2ba14-4c8c-41de-82bb-48b05304d090]: (4, ('tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\ntcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:41:11 user nova-compute[71317]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)
Apr 14 04:41:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:41:11 user nova-compute[71317]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 (non-flash)
Apr 14 04:41:11 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84b4215480>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b49739e6-245f-4b30-888a-4caf6f85e2fc]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2eb37ea-6444-4fd1-b7ae-44fcd2adb57d]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a99bbef-d9f7-4425-9fd2-7232e389098d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.045s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[babe708e-4c54-4036-aed2-738259501c25]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23362643134306261', '/dev/disk/by-id/scsi-1SCST_BIOf2d54187-3bd140ba6a79949a', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_f2d54187') {{(pid=81937) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[67535d7d-7b6e-44a0-b739-5b114c1f33d3]: (4, None) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[42c100e2-0181-4f6f-b34a-3491d9266ad6]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:41:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --logout" returned: 0 in 0.054s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[47f56f51-da42-4cf9-8607-e803d0c270ea]: (4, ('Logging out of session [sid: 51, target: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1, portal: 172.16.0.220,3260]\nLogout of [sid: 51, target: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 51, target: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1, portal: 172.16.0.220,3260]
Apr 14 04:41:12 user nova-compute[71317]: Logout of [sid: 51, target: iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1, portal: 172.16.0.220,3260] successful.
Apr 14 04:41:12 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:350da73c-f3b8-4777-b0d2-56bc054366d1 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[08847c15-68d1-437d-bdf1-8826dc3ebcfc]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.307s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== disconnect_volume: return (308ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:41:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Deleting instance files /opt/stack/data/nova/instances/9d93dc7e-d497-4d8f-9ade-c508170f6121_del
Apr 14 04:41:12 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Deletion of /opt/stack/data/nova/instances/9d93dc7e-d497-4d8f-9ade-c508170f6121_del complete
Apr 14 04:41:12 user nova-compute[71317]: INFO nova.compute.manager [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Took 1.18 seconds to destroy the instance on the hypervisor.
Apr 14 04:41:12 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-2df29ad9-e134-465f-b5ec-a084a5d922c1 req-51b173ad-cc88-459d-b734-eb01ec9f1c58 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Received event network-changed-2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-2df29ad9-e134-465f-b5ec-a084a5d922c1 req-51b173ad-cc88-459d-b734-eb01ec9f1c58 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Refreshing instance network info cache due to event network-changed-2fcf791e-22d7-4a96-894c-70900772a5ff. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2df29ad9-e134-465f-b5ec-a084a5d922c1 req-51b173ad-cc88-459d-b734-eb01ec9f1c58 service nova] Acquiring lock "refresh_cache-9d93dc7e-d497-4d8f-9ade-c508170f6121" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2df29ad9-e134-465f-b5ec-a084a5d922c1 req-51b173ad-cc88-459d-b734-eb01ec9f1c58 service nova] Acquired lock "refresh_cache-9d93dc7e-d497-4d8f-9ade-c508170f6121" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG nova.network.neutron [req-2df29ad9-e134-465f-b5ec-a084a5d922c1 req-51b173ad-cc88-459d-b734-eb01ec9f1c58 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Refreshing network info cache for port 2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:41:12 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:41:12 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Took 0.51 seconds to deallocate network for instance.
Apr 14 04:41:12 user nova-compute[71317]: DEBUG nova.compute.manager [req-a2d1b32b-c153-424d-8738-17b715b97911 req-0dfeb969-f026-4a84-b422-4b7f66028761 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Received event network-vif-deleted-2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG nova.network.neutron [req-2df29ad9-e134-465f-b5ec-a084a5d922c1 req-51b173ad-cc88-459d-b734-eb01ec9f1c58 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Updated VIF entry in instance network info cache for port 2fcf791e-22d7-4a96-894c-70900772a5ff. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG nova.network.neutron [req-2df29ad9-e134-465f-b5ec-a084a5d922c1 req-51b173ad-cc88-459d-b734-eb01ec9f1c58 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Updating instance_info_cache with network_info: [{"id": "2fcf791e-22d7-4a96-894c-70900772a5ff", "address": "fa:16:3e:67:49:c3", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2fcf791e-22", "ovs_interfaceid": "2fcf791e-22d7-4a96-894c-70900772a5ff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2df29ad9-e134-465f-b5ec-a084a5d922c1 req-51b173ad-cc88-459d-b734-eb01ec9f1c58 service nova] Releasing lock "refresh_cache-9d93dc7e-d497-4d8f-9ade-c508170f6121" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-ab11b844-287c-47c9-ac45-94e25116a976 req-73a8480c-6e3c-4314-8fd1-b7d509f9cab6 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Received event network-vif-plugged-2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ab11b844-287c-47c9-ac45-94e25116a976 req-73a8480c-6e3c-4314-8fd1-b7d509f9cab6 service nova] Acquiring lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ab11b844-287c-47c9-ac45-94e25116a976 req-73a8480c-6e3c-4314-8fd1-b7d509f9cab6 service nova] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ab11b844-287c-47c9-ac45-94e25116a976 req-73a8480c-6e3c-4314-8fd1-b7d509f9cab6 service nova] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-ab11b844-287c-47c9-ac45-94e25116a976 req-73a8480c-6e3c-4314-8fd1-b7d509f9cab6 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] No waiting events found dispatching network-vif-plugged-2fcf791e-22d7-4a96-894c-70900772a5ff {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:41:13 user nova-compute[71317]: WARNING nova.compute.manager [req-ab11b844-287c-47c9-ac45-94e25116a976 req-73a8480c-6e3c-4314-8fd1-b7d509f9cab6 service nova] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Received unexpected event network-vif-plugged-2fcf791e-22d7-4a96-894c-70900772a5ff for instance with vm_state active and task_state deleting.
Apr 14 04:41:13 user nova-compute[71317]: INFO nova.compute.manager [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Took 0.82 seconds to detach 1 volumes for instance.
Apr 14 04:41:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.144s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:13 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Deleted allocations for instance 9d93dc7e-d497-4d8f-9ade-c508170f6121
Apr 14 04:41:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-d086675a-6290-4d70-9c71-f2762313b8cd tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "9d93dc7e-d497-4d8f-9ade-c508170f6121" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.844s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:41:16 user nova-compute[71317]: INFO nova.compute.claims [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Claim successful on node user
Apr 14 04:41:16 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.209s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG nova.network.neutron [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:41:16 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:41:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG nova.policy [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7cab2461ab0d4eb08b62feaf6396bc87', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '601e2806907a48d3995cc90b54da3c8c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:41:16 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Creating image(s)
Apr 14 04:41:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "/opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "/opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "/opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.005s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:16 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk 1073741824" returned: 0 in 0.051s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.201s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Checking if we can resize image /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG nova.network.neutron [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Successfully created port: 6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Cannot resize image /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'migration_context' on Instance uuid 3d8009dc-c463-41f2-8b36-21862b7ad5ca {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Ensure instance console log exists: /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Successfully updated port: 6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquired lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-a33ac9c4-486d-4771-9d69-23b58d22507e req-3186419d-b374-4fed-ad26-37acc9e9c96c service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Received event network-changed-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-a33ac9c4-486d-4771-9d69-23b58d22507e req-3186419d-b374-4fed-ad26-37acc9e9c96c service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Refreshing instance network info cache due to event network-changed-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a33ac9c4-486d-4771-9d69-23b58d22507e req-3186419d-b374-4fed-ad26-37acc9e9c96c service nova] Acquiring lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.network.neutron [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Updating instance_info_cache with network_info: [{"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Releasing lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Instance network_info: |[{"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a33ac9c4-486d-4771-9d69-23b58d22507e req-3186419d-b374-4fed-ad26-37acc9e9c96c service nova] Acquired lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.network.neutron [req-a33ac9c4-486d-4771-9d69-23b58d22507e req-3186419d-b374-4fed-ad26-37acc9e9c96c service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Refreshing network info cache for port 6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Start _get_guest_xml network_info=[{"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:41:18 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:41:18 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:41:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-610195412',display_name='tempest-AttachVolumeNegativeTest-server-610195412',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-610195412',id=43,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC3TphBlHcrFQUWfHI9Rd+a22lD9cX0GeecKjEQEW/ftTRl57yNsm3ZsNJGGAxbXJX4cCwx+4dgcJcdGA6/AzTSvdleEod5ijJD25/l5NF3G7sPYAP/bwHLaLqv+XnXDSQ==',key_name='tempest-keypair-2017831093',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='601e2806907a48d3995cc90b54da3c8c',ramdisk_id='',reservation_id='r-lm0pepyf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1197484356',owner_user_name='tempest-AttachVolumeNegativeTest-1197484356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:41:17Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7cab2461ab0d4eb08b62feaf6396bc87',uuid=3d8009dc-c463-41f2-8b36-21862b7ad5ca,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converting VIF {"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1b:b5:53,bridge_name='br-int',has_traffic_filtering=True,id=6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9b1d34-ea') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.objects.instance [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'pci_devices' on Instance uuid 3d8009dc-c463-41f2-8b36-21862b7ad5ca {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:41:18 user nova-compute[71317]:   <uuid>3d8009dc-c463-41f2-8b36-21862b7ad5ca</uuid>
Apr 14 04:41:18 user nova-compute[71317]:   <name>instance-0000002b</name>
Apr 14 04:41:18 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:41:18 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:41:18 user nova-compute[71317]:   <metadata>
Apr 14 04:41:18 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:41:18 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:41:18 user nova-compute[71317]:       <nova:name>tempest-AttachVolumeNegativeTest-server-610195412</nova:name>
Apr 14 04:41:18 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:41:18</nova:creationTime>
Apr 14 04:41:18 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:41:18 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:41:18 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:41:18 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:41:18 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:41:18 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:41:18 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:41:18 user nova-compute[71317]:       <nova:owner>
Apr 14 04:41:18 user nova-compute[71317]:         <nova:user uuid="7cab2461ab0d4eb08b62feaf6396bc87">tempest-AttachVolumeNegativeTest-1197484356-project-member</nova:user>
Apr 14 04:41:18 user nova-compute[71317]:         <nova:project uuid="601e2806907a48d3995cc90b54da3c8c">tempest-AttachVolumeNegativeTest-1197484356</nova:project>
Apr 14 04:41:18 user nova-compute[71317]:       </nova:owner>
Apr 14 04:41:18 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:41:18 user nova-compute[71317]:       <nova:ports>
Apr 14 04:41:18 user nova-compute[71317]:         <nova:port uuid="6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68">
Apr 14 04:41:18 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Apr 14 04:41:18 user nova-compute[71317]:         </nova:port>
Apr 14 04:41:18 user nova-compute[71317]:       </nova:ports>
Apr 14 04:41:18 user nova-compute[71317]:     </nova:instance>
Apr 14 04:41:18 user nova-compute[71317]:   </metadata>
Apr 14 04:41:18 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:41:18 user nova-compute[71317]:     <system>
Apr 14 04:41:18 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:41:18 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:41:18 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:41:18 user nova-compute[71317]:       <entry name="serial">3d8009dc-c463-41f2-8b36-21862b7ad5ca</entry>
Apr 14 04:41:18 user nova-compute[71317]:       <entry name="uuid">3d8009dc-c463-41f2-8b36-21862b7ad5ca</entry>
Apr 14 04:41:18 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:41:18 user nova-compute[71317]:     </system>
Apr 14 04:41:18 user nova-compute[71317]:   </sysinfo>
Apr 14 04:41:18 user nova-compute[71317]:   <os>
Apr 14 04:41:18 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:41:18 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:41:18 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:41:18 user nova-compute[71317]:   </os>
Apr 14 04:41:18 user nova-compute[71317]:   <features>
Apr 14 04:41:18 user nova-compute[71317]:     <acpi/>
Apr 14 04:41:18 user nova-compute[71317]:     <apic/>
Apr 14 04:41:18 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:41:18 user nova-compute[71317]:   </features>
Apr 14 04:41:18 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:41:18 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:41:18 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:41:18 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:41:18 user nova-compute[71317]:   </clock>
Apr 14 04:41:18 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:41:18 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:41:18 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:41:18 user nova-compute[71317]:   </cpu>
Apr 14 04:41:18 user nova-compute[71317]:   <devices>
Apr 14 04:41:18 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:41:18 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:41:18 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk"/>
Apr 14 04:41:18 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:41:18 user nova-compute[71317]:     </disk>
Apr 14 04:41:18 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:41:18 user nova-compute[71317]:       <mac address="fa:16:3e:1b:b5:53"/>
Apr 14 04:41:18 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:41:18 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:41:18 user nova-compute[71317]:       <target dev="tap6d9b1d34-ea"/>
Apr 14 04:41:18 user nova-compute[71317]:     </interface>
Apr 14 04:41:18 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:41:18 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/console.log" append="off"/>
Apr 14 04:41:18 user nova-compute[71317]:     </serial>
Apr 14 04:41:18 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:41:18 user nova-compute[71317]:     <video>
Apr 14 04:41:18 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:41:18 user nova-compute[71317]:     </video>
Apr 14 04:41:18 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:41:18 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:41:18 user nova-compute[71317]:     </rng>
Apr 14 04:41:18 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:41:18 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:41:18 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:41:18 user nova-compute[71317]:     </memballoon>
Apr 14 04:41:18 user nova-compute[71317]:   </devices>
Apr 14 04:41:18 user nova-compute[71317]: </domain>
Apr 14 04:41:18 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:41:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-610195412',display_name='tempest-AttachVolumeNegativeTest-server-610195412',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-610195412',id=43,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC3TphBlHcrFQUWfHI9Rd+a22lD9cX0GeecKjEQEW/ftTRl57yNsm3ZsNJGGAxbXJX4cCwx+4dgcJcdGA6/AzTSvdleEod5ijJD25/l5NF3G7sPYAP/bwHLaLqv+XnXDSQ==',key_name='tempest-keypair-2017831093',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='601e2806907a48d3995cc90b54da3c8c',ramdisk_id='',reservation_id='r-lm0pepyf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1197484356',owner_user_name='tempest-AttachVolumeNegativeTest-1197484356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:41:17Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7cab2461ab0d4eb08b62feaf6396bc87',uuid=3d8009dc-c463-41f2-8b36-21862b7ad5ca,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converting VIF {"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1b:b5:53,bridge_name='br-int',has_traffic_filtering=True,id=6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9b1d34-ea') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG os_vif [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:1b:b5:53,bridge_name='br-int',has_traffic_filtering=True,id=6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9b1d34-ea') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap6d9b1d34-ea, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap6d9b1d34-ea, col_values=(('external_ids', {'iface-id': '6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:1b:b5:53', 'vm-uuid': '3d8009dc-c463-41f2-8b36-21862b7ad5ca'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:18 user nova-compute[71317]: INFO os_vif [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:1b:b5:53,bridge_name='br-int',has_traffic_filtering=True,id=6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9b1d34-ea')
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:41:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No VIF found with MAC fa:16:3e:1b:b5:53, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:41:19 user nova-compute[71317]: DEBUG nova.network.neutron [req-a33ac9c4-486d-4771-9d69-23b58d22507e req-3186419d-b374-4fed-ad26-37acc9e9c96c service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Updated VIF entry in instance network info cache for port 6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:41:19 user nova-compute[71317]: DEBUG nova.network.neutron [req-a33ac9c4-486d-4771-9d69-23b58d22507e req-3186419d-b374-4fed-ad26-37acc9e9c96c service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Updating instance_info_cache with network_info: [{"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:41:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a33ac9c4-486d-4771-9d69-23b58d22507e req-3186419d-b374-4fed-ad26-37acc9e9c96c service nova] Releasing lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:41:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-58a9cffa-dd59-4076-926f-09db4fe494b8 req-d05fb58c-ee07-427a-9be2-7cac9d135449 service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Received event network-vif-plugged-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:41:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-58a9cffa-dd59-4076-926f-09db4fe494b8 req-d05fb58c-ee07-427a-9be2-7cac9d135449 service nova] Acquiring lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-58a9cffa-dd59-4076-926f-09db4fe494b8 req-d05fb58c-ee07-427a-9be2-7cac9d135449 service nova] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-58a9cffa-dd59-4076-926f-09db4fe494b8 req-d05fb58c-ee07-427a-9be2-7cac9d135449 service nova] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:20 user nova-compute[71317]: DEBUG nova.compute.manager [req-58a9cffa-dd59-4076-926f-09db4fe494b8 req-d05fb58c-ee07-427a-9be2-7cac9d135449 service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] No waiting events found dispatching network-vif-plugged-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:41:20 user nova-compute[71317]: WARNING nova.compute.manager [req-58a9cffa-dd59-4076-926f-09db4fe494b8 req-d05fb58c-ee07-427a-9be2-7cac9d135449 service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Received unexpected event network-vif-plugged-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 for instance with vm_state building and task_state spawning.
Apr 14 04:41:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447282.0189095, 3d8009dc-c463-41f2-8b36-21862b7ad5ca => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:41:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] VM Resumed (Lifecycle Event)
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:41:22 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Instance spawned successfully.
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:41:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447282.0206485, 3d8009dc-c463-41f2-8b36-21862b7ad5ca => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:41:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] VM Started (Lifecycle Event)
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:41:22 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:41:22 user nova-compute[71317]: INFO nova.compute.manager [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Took 5.24 seconds to spawn the instance on the hypervisor.
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-0a141052-1679-43da-b395-70bc67ee60b4 req-31df3052-f616-4811-b9fa-350de31e3a3d service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Received event network-vif-plugged-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0a141052-1679-43da-b395-70bc67ee60b4 req-31df3052-f616-4811-b9fa-350de31e3a3d service nova] Acquiring lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0a141052-1679-43da-b395-70bc67ee60b4 req-31df3052-f616-4811-b9fa-350de31e3a3d service nova] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0a141052-1679-43da-b395-70bc67ee60b4 req-31df3052-f616-4811-b9fa-350de31e3a3d service nova] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-0a141052-1679-43da-b395-70bc67ee60b4 req-31df3052-f616-4811-b9fa-350de31e3a3d service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] No waiting events found dispatching network-vif-plugged-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:41:22 user nova-compute[71317]: WARNING nova.compute.manager [req-0a141052-1679-43da-b395-70bc67ee60b4 req-31df3052-f616-4811-b9fa-350de31e3a3d service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Received unexpected event network-vif-plugged-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 for instance with vm_state active and task_state None.
Apr 14 04:41:22 user nova-compute[71317]: INFO nova.compute.manager [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Took 5.80 seconds to build instance.
Apr 14 04:41:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-58c09e33-2c04-4e2c-ba75-9fb21c30d050 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 5.917s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:26 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447271.711008, 9d93dc7e-d497-4d8f-9ade-c508170f6121 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:41:26 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] VM Stopped (Lifecycle Event)
Apr 14 04:41:26 user nova-compute[71317]: DEBUG nova.compute.manager [None req-dca4d832-0360-4e50-9daf-c82b18d65e35 None None] [instance: 9d93dc7e-d497-4d8f-9ade-c508170f6121] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:41:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:31 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:38 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:41:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:41:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:43 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:41:43 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:41:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:44 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:41:44 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:41:44 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:41:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:41:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:41:44 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:41:44 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lazy-loading 'info_cache' on Instance uuid 49c7ae26-8d9e-45ac-a189-3ee84e015b27 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:41:44 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updating instance_info_cache with network_info: [{"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.171", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:41:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:41:44 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:41:46 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:46 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:41:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:41:51 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:41:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:51 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:41:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:41:51 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:41:51 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:41:51 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:41:51 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:41:51 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8537MB free_disk=26.355186462402344GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:41:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:41:51 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:41:52 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 49c7ae26-8d9e-45ac-a189-3ee84e015b27 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:41:52 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3d8009dc-c463-41f2-8b36-21862b7ad5ca actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:41:52 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:41:52 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=768MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:41:52 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:41:52 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:41:52 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:41:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.218s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:41:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:53 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:41:53 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:41:53 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:41:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:41:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "77f6bb83-54ab-485e-8943-742f2f576607" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:00 user nova-compute[71317]: DEBUG nova.compute.manager [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:42:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:00 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:42:00 user nova-compute[71317]: INFO nova.compute.claims [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Claim successful on node user
Apr 14 04:42:01 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.253s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG nova.network.neutron [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:42:01 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:42:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG nova.policy [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '67d6c699fc524554a6f4983af91d58fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c10c8cf348e04e44afbaae93a4b60e36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:42:01 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Creating image(s)
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "/opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "/opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "/opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.145s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk 1073741824" returned: 0 in 0.052s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.193s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.159s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Checking if we can resize image /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:42:01 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Cannot resize image /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG nova.objects.instance [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'migration_context' on Instance uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Ensure instance console log exists: /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG nova.network.neutron [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Successfully created port: 440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG nova.network.neutron [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Successfully updated port: 440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquired lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG nova.network.neutron [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-63eea503-5cd4-4f59-8209-dcb2f304d607 req-ed4024d8-8b7f-41ea-9992-8a53b6179384 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-changed-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-63eea503-5cd4-4f59-8209-dcb2f304d607 req-ed4024d8-8b7f-41ea-9992-8a53b6179384 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Refreshing instance network info cache due to event network-changed-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-63eea503-5cd4-4f59-8209-dcb2f304d607 req-ed4024d8-8b7f-41ea-9992-8a53b6179384 service nova] Acquiring lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:42:02 user nova-compute[71317]: DEBUG nova.network.neutron [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.network.neutron [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updating instance_info_cache with network_info: [{"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Releasing lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Instance network_info: |[{"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-63eea503-5cd4-4f59-8209-dcb2f304d607 req-ed4024d8-8b7f-41ea-9992-8a53b6179384 service nova] Acquired lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.network.neutron [req-63eea503-5cd4-4f59-8209-dcb2f304d607 req-ed4024d8-8b7f-41ea-9992-8a53b6179384 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Refreshing network info cache for port 440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Start _get_guest_xml network_info=[{"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:42:03 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:42:03 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:42:00Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1249800375',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1249800375',id=44,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH6fraox0ufDVlTeIxqf3LA1Jqk/6GB3uKQGBsTlrxVUqCfAkV+hrTL0aqpPNJ0Y4OrCsZ9AqcY4PXRY4CmEWIke2qyPR134zIiDdDnr3VeL889NFNXZb2Fua8pApWkvww==',key_name='tempest-keypair-1996810505',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c10c8cf348e04e44afbaae93a4b60e36',ramdisk_id='',reservation_id='r-4s0bcxm4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-2141578557',owner_user_name='tempest-AttachVolumeShelveTestJSON-2141578557-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:42:01Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='67d6c699fc524554a6f4983af91d58fc',uuid=77f6bb83-54ab-485e-8943-742f2f576607,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converting VIF {"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.objects.instance [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'pci_devices' on Instance uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:42:03 user nova-compute[71317]:   <uuid>77f6bb83-54ab-485e-8943-742f2f576607</uuid>
Apr 14 04:42:03 user nova-compute[71317]:   <name>instance-0000002c</name>
Apr 14 04:42:03 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:42:03 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:42:03 user nova-compute[71317]:   <metadata>
Apr 14 04:42:03 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:42:03 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:42:03 user nova-compute[71317]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-1249800375</nova:name>
Apr 14 04:42:03 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:42:03</nova:creationTime>
Apr 14 04:42:03 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:42:03 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:42:03 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:42:03 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:42:03 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:42:03 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:42:03 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:42:03 user nova-compute[71317]:       <nova:owner>
Apr 14 04:42:03 user nova-compute[71317]:         <nova:user uuid="67d6c699fc524554a6f4983af91d58fc">tempest-AttachVolumeShelveTestJSON-2141578557-project-member</nova:user>
Apr 14 04:42:03 user nova-compute[71317]:         <nova:project uuid="c10c8cf348e04e44afbaae93a4b60e36">tempest-AttachVolumeShelveTestJSON-2141578557</nova:project>
Apr 14 04:42:03 user nova-compute[71317]:       </nova:owner>
Apr 14 04:42:03 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:42:03 user nova-compute[71317]:       <nova:ports>
Apr 14 04:42:03 user nova-compute[71317]:         <nova:port uuid="440e7ef0-4f4c-4923-ab57-8e1ba56dbea3">
Apr 14 04:42:03 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Apr 14 04:42:03 user nova-compute[71317]:         </nova:port>
Apr 14 04:42:03 user nova-compute[71317]:       </nova:ports>
Apr 14 04:42:03 user nova-compute[71317]:     </nova:instance>
Apr 14 04:42:03 user nova-compute[71317]:   </metadata>
Apr 14 04:42:03 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:42:03 user nova-compute[71317]:     <system>
Apr 14 04:42:03 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:42:03 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:42:03 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:42:03 user nova-compute[71317]:       <entry name="serial">77f6bb83-54ab-485e-8943-742f2f576607</entry>
Apr 14 04:42:03 user nova-compute[71317]:       <entry name="uuid">77f6bb83-54ab-485e-8943-742f2f576607</entry>
Apr 14 04:42:03 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:42:03 user nova-compute[71317]:     </system>
Apr 14 04:42:03 user nova-compute[71317]:   </sysinfo>
Apr 14 04:42:03 user nova-compute[71317]:   <os>
Apr 14 04:42:03 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:42:03 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:42:03 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:42:03 user nova-compute[71317]:   </os>
Apr 14 04:42:03 user nova-compute[71317]:   <features>
Apr 14 04:42:03 user nova-compute[71317]:     <acpi/>
Apr 14 04:42:03 user nova-compute[71317]:     <apic/>
Apr 14 04:42:03 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:42:03 user nova-compute[71317]:   </features>
Apr 14 04:42:03 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:42:03 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:42:03 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:42:03 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:42:03 user nova-compute[71317]:   </clock>
Apr 14 04:42:03 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:42:03 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:42:03 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:42:03 user nova-compute[71317]:   </cpu>
Apr 14 04:42:03 user nova-compute[71317]:   <devices>
Apr 14 04:42:03 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:42:03 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:42:03 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk"/>
Apr 14 04:42:03 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:42:03 user nova-compute[71317]:     </disk>
Apr 14 04:42:03 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:42:03 user nova-compute[71317]:       <mac address="fa:16:3e:6b:b2:cc"/>
Apr 14 04:42:03 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:42:03 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:42:03 user nova-compute[71317]:       <target dev="tap440e7ef0-4f"/>
Apr 14 04:42:03 user nova-compute[71317]:     </interface>
Apr 14 04:42:03 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:42:03 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/console.log" append="off"/>
Apr 14 04:42:03 user nova-compute[71317]:     </serial>
Apr 14 04:42:03 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:42:03 user nova-compute[71317]:     <video>
Apr 14 04:42:03 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:42:03 user nova-compute[71317]:     </video>
Apr 14 04:42:03 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:42:03 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:42:03 user nova-compute[71317]:     </rng>
Apr 14 04:42:03 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:42:03 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:42:03 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:42:03 user nova-compute[71317]:     </memballoon>
Apr 14 04:42:03 user nova-compute[71317]:   </devices>
Apr 14 04:42:03 user nova-compute[71317]: </domain>
Apr 14 04:42:03 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:42:00Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1249800375',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1249800375',id=44,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH6fraox0ufDVlTeIxqf3LA1Jqk/6GB3uKQGBsTlrxVUqCfAkV+hrTL0aqpPNJ0Y4OrCsZ9AqcY4PXRY4CmEWIke2qyPR134zIiDdDnr3VeL889NFNXZb2Fua8pApWkvww==',key_name='tempest-keypair-1996810505',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c10c8cf348e04e44afbaae93a4b60e36',ramdisk_id='',reservation_id='r-4s0bcxm4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-2141578557',owner_user_name='tempest-AttachVolumeShelveTestJSON-2141578557-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:42:01Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='67d6c699fc524554a6f4983af91d58fc',uuid=77f6bb83-54ab-485e-8943-742f2f576607,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converting VIF {"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG os_vif [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap440e7ef0-4f, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap440e7ef0-4f, col_values=(('external_ids', {'iface-id': '440e7ef0-4f4c-4923-ab57-8e1ba56dbea3', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6b:b2:cc', 'vm-uuid': '77f6bb83-54ab-485e-8943-742f2f576607'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:03 user nova-compute[71317]: INFO os_vif [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f')
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] No VIF found with MAC fa:16:3e:6b:b2:cc, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:42:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:04 user nova-compute[71317]: DEBUG nova.network.neutron [req-63eea503-5cd4-4f59-8209-dcb2f304d607 req-ed4024d8-8b7f-41ea-9992-8a53b6179384 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updated VIF entry in instance network info cache for port 440e7ef0-4f4c-4923-ab57-8e1ba56dbea3. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:42:04 user nova-compute[71317]: DEBUG nova.network.neutron [req-63eea503-5cd4-4f59-8209-dcb2f304d607 req-ed4024d8-8b7f-41ea-9992-8a53b6179384 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updating instance_info_cache with network_info: [{"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:42:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-63eea503-5cd4-4f59-8209-dcb2f304d607 req-ed4024d8-8b7f-41ea-9992-8a53b6179384 service nova] Releasing lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:42:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:05 user nova-compute[71317]: DEBUG nova.compute.manager [req-a3ad2674-35d1-4c68-9d3b-f5ae3130c423 req-f810c284-dd39-4b07-96cb-bc11cda9873d service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:42:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a3ad2674-35d1-4c68-9d3b-f5ae3130c423 req-f810c284-dd39-4b07-96cb-bc11cda9873d service nova] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a3ad2674-35d1-4c68-9d3b-f5ae3130c423 req-f810c284-dd39-4b07-96cb-bc11cda9873d service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a3ad2674-35d1-4c68-9d3b-f5ae3130c423 req-f810c284-dd39-4b07-96cb-bc11cda9873d service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:05 user nova-compute[71317]: DEBUG nova.compute.manager [req-a3ad2674-35d1-4c68-9d3b-f5ae3130c423 req-f810c284-dd39-4b07-96cb-bc11cda9873d service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] No waiting events found dispatching network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:42:05 user nova-compute[71317]: WARNING nova.compute.manager [req-a3ad2674-35d1-4c68-9d3b-f5ae3130c423 req-f810c284-dd39-4b07-96cb-bc11cda9873d service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received unexpected event network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 for instance with vm_state building and task_state spawning.
Apr 14 04:42:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447327.0635383, 77f6bb83-54ab-485e-8943-742f2f576607 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:42:07 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] VM Resumed (Lifecycle Event)
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:42:07 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Instance spawned successfully.
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:42:07 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447327.0650845, 77f6bb83-54ab-485e-8943-742f2f576607 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:42:07 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] VM Started (Lifecycle Event)
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-b7b06080-7c3f-4eb7-8acf-0526420aabd4 req-c8f65058-ae84-4501-8356-ad0a6cea1084 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b7b06080-7c3f-4eb7-8acf-0526420aabd4 req-c8f65058-ae84-4501-8356-ad0a6cea1084 service nova] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b7b06080-7c3f-4eb7-8acf-0526420aabd4 req-c8f65058-ae84-4501-8356-ad0a6cea1084 service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b7b06080-7c3f-4eb7-8acf-0526420aabd4 req-c8f65058-ae84-4501-8356-ad0a6cea1084 service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-b7b06080-7c3f-4eb7-8acf-0526420aabd4 req-c8f65058-ae84-4501-8356-ad0a6cea1084 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] No waiting events found dispatching network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:42:07 user nova-compute[71317]: WARNING nova.compute.manager [req-b7b06080-7c3f-4eb7-8acf-0526420aabd4 req-c8f65058-ae84-4501-8356-ad0a6cea1084 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received unexpected event network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 for instance with vm_state building and task_state spawning.
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:42:07 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:42:07 user nova-compute[71317]: INFO nova.compute.manager [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Took 5.86 seconds to spawn the instance on the hypervisor.
Apr 14 04:42:07 user nova-compute[71317]: DEBUG nova.compute.manager [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:42:07 user nova-compute[71317]: INFO nova.compute.manager [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Took 6.49 seconds to build instance.
Apr 14 04:42:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-de3873ba-256d-48aa-bde0-98ca7ca1164f tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "77f6bb83-54ab-485e-8943-742f2f576607" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.585s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5d8cbe5e-d276-45b4-8d34-23cf4cc14c47 req-9709dfde-0106-46e8-91df-20b8f19fc02e service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Received event network-changed-f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-5d8cbe5e-d276-45b4-8d34-23cf4cc14c47 req-9709dfde-0106-46e8-91df-20b8f19fc02e service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Refreshing instance network info cache due to event network-changed-f204cf26-ba59-48a1-b08e-6d8915075961. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5d8cbe5e-d276-45b4-8d34-23cf4cc14c47 req-9709dfde-0106-46e8-91df-20b8f19fc02e service nova] Acquiring lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5d8cbe5e-d276-45b4-8d34-23cf4cc14c47 req-9709dfde-0106-46e8-91df-20b8f19fc02e service nova] Acquired lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG nova.network.neutron [req-5d8cbe5e-d276-45b4-8d34-23cf4cc14c47 req-9709dfde-0106-46e8-91df-20b8f19fc02e service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Refreshing network info cache for port f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:08 user nova-compute[71317]: INFO nova.compute.manager [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Terminating instance
Apr 14 04:42:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG nova.network.neutron [req-5d8cbe5e-d276-45b4-8d34-23cf4cc14c47 req-9709dfde-0106-46e8-91df-20b8f19fc02e service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updated VIF entry in instance network info cache for port f204cf26-ba59-48a1-b08e-6d8915075961. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG nova.network.neutron [req-5d8cbe5e-d276-45b4-8d34-23cf4cc14c47 req-9709dfde-0106-46e8-91df-20b8f19fc02e service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updating instance_info_cache with network_info: [{"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-5d8cbe5e-d276-45b4-8d34-23cf4cc14c47 req-9709dfde-0106-46e8-91df-20b8f19fc02e service nova] Releasing lock "refresh_cache-49c7ae26-8d9e-45ac-a189-3ee84e015b27" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:42:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Instance destroyed successfully.
Apr 14 04:42:09 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lazy-loading 'resources' on Instance uuid 49c7ae26-8d9e-45ac-a189-3ee84e015b27 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:37:17Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1983263653',display_name='tempest-TestVolumeBootPattern-server-1983263653',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1983263653',id=41,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFZO4o4kQgpEvaFt8iKOk4kfW7YF9jZ6XS2jVS2UAmaxmR76rP/jf7a3pf4xOyQu5V8HIaLR//SNI2rVtnFikRInNPtRWTjIBvrk9d4sXMIgR6FP64tMb+FLBXcO2su56A==',key_name='tempest-TestVolumeBootPattern-1640497739',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:37:27Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='693d95858c624feaa60114aa89082186',ramdisk_id='',reservation_id='r-mzzgtbac',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-1780160917',owner_user_name='tempest-TestVolumeBootPattern-1780160917-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:37:28Z,user_data=None,user_id='bcf79de7da27495dbfd14c556b5b54c7',uuid=49c7ae26-8d9e-45ac-a189-3ee84e015b27,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.171", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converting VIF {"id": "f204cf26-ba59-48a1-b08e-6d8915075961", "address": "fa:16:3e:ff:ad:63", "network": {"id": "e20e0d94-3b17-4872-b774-3a724d3a253b", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-630044086-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.171", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "693d95858c624feaa60114aa89082186", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf204cf26-ba", "ovs_interfaceid": "f204cf26-ba59-48a1-b08e-6d8915075961", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ff:ad:63,bridge_name='br-int',has_traffic_filtering=True,id=f204cf26-ba59-48a1-b08e-6d8915075961,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf204cf26-ba') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_vif [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:ff:ad:63,bridge_name='br-int',has_traffic_filtering=True,id=f204cf26-ba59-48a1-b08e-6d8915075961,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf204cf26-ba') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapf204cf26-ba, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:42:09 user nova-compute[71317]: INFO os_vif [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:ff:ad:63,bridge_name='br-int',has_traffic_filtering=True,id=f204cf26-ba59-48a1-b08e-6d8915075961,network=Network(e20e0d94-3b17-4872-b774-3a724d3a253b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf204cf26-ba')
Apr 14 04:42:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tLxSpwaP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:42:09 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[835a8d35-9f38-4625-931e-c0149b180baa]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:42:09 user nova-compute[71317]: No targets found.
Apr 14 04:42:09 user nova-compute[71317]: iSNS:
Apr 14 04:42:09 user nova-compute[71317]: No targets found.
Apr 14 04:42:09 user nova-compute[71317]: STATIC:
Apr 14 04:42:09 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75
Apr 14 04:42:09 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:42:09 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:42:09 user nova-compute[71317]: FIRMWARE:
Apr 14 04:42:09 user nova-compute[71317]: No targets found.
Apr 14 04:42:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:42:09 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:42:09 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c911de1-bf0f-410b-a180-13def3f6ac8d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d80a6a75-5044-421d-b6f2-8c45a69b5d94]: (4, ('tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:42:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsi session list stdout=tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 (non-flash)
Apr 14 04:42:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84902a1f30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75'): ({'sdf'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Removing single pathed devices sdf {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a55a7d07-539a-42ef-94e3-f2f8afa13192]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c98cf94-db07-4097-8e2a-566c7909f97d]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Flushing IO for device /dev/sdf {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8714e842-82aa-48fa-a592-1d448c606742]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.055s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7eed91d1-d8bf-4fb2-957b-11ef4a43e79d]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] SCSI volumes sdf have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[179757c3-216f-4664-8336-2251860c6875]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --logout" returned: 0 in 0.057s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae1f6666-b07c-4416-843c-5798bc2e73a7]: (4, ('Logging out of session [sid: 50, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260]\nLogout of [sid: 50, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 50, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260]
Apr 14 04:42:09 user nova-compute[71317]: Logout of [sid: 50, target: iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75, portal: 172.16.0.220,3260] successful.
Apr 14 04:42:09 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:495f173c-6873-42b6-a0fe-42da67f3aa75 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0af235ea-cc26-40a5-a46a-d65fe9b89814]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.342s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] <== disconnect_volume: return (344ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:42:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:42:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Deleting instance files /opt/stack/data/nova/instances/49c7ae26-8d9e-45ac-a189-3ee84e015b27_del
Apr 14 04:42:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Deletion of /opt/stack/data/nova/instances/49c7ae26-8d9e-45ac-a189-3ee84e015b27_del complete
Apr 14 04:42:10 user nova-compute[71317]: INFO nova.compute.manager [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Took 1.28 seconds to destroy the instance on the hypervisor.
Apr 14 04:42:10 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:42:10 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Took 0.58 seconds to deallocate network for instance.
Apr 14 04:42:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-1dbe70c4-bfc0-4c90-9ba6-745cad69db74 req-30770ff6-58ea-44c4-8720-fd8de5ef47d2 service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Received event network-vif-deleted-f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-f0fb6cc1-ed2a-4588-80b4-f134ee28b855 req-be34bf64-fc01-4aba-892a-1c3458d11f5a service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Received event network-vif-unplugged-f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0fb6cc1-ed2a-4588-80b4-f134ee28b855 req-be34bf64-fc01-4aba-892a-1c3458d11f5a service nova] Acquiring lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0fb6cc1-ed2a-4588-80b4-f134ee28b855 req-be34bf64-fc01-4aba-892a-1c3458d11f5a service nova] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0fb6cc1-ed2a-4588-80b4-f134ee28b855 req-be34bf64-fc01-4aba-892a-1c3458d11f5a service nova] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-f0fb6cc1-ed2a-4588-80b4-f134ee28b855 req-be34bf64-fc01-4aba-892a-1c3458d11f5a service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] No waiting events found dispatching network-vif-unplugged-f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-f0fb6cc1-ed2a-4588-80b4-f134ee28b855 req-be34bf64-fc01-4aba-892a-1c3458d11f5a service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Received event network-vif-unplugged-f204cf26-ba59-48a1-b08e-6d8915075961 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-f0fb6cc1-ed2a-4588-80b4-f134ee28b855 req-be34bf64-fc01-4aba-892a-1c3458d11f5a service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Received event network-vif-plugged-f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0fb6cc1-ed2a-4588-80b4-f134ee28b855 req-be34bf64-fc01-4aba-892a-1c3458d11f5a service nova] Acquiring lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0fb6cc1-ed2a-4588-80b4-f134ee28b855 req-be34bf64-fc01-4aba-892a-1c3458d11f5a service nova] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0fb6cc1-ed2a-4588-80b4-f134ee28b855 req-be34bf64-fc01-4aba-892a-1c3458d11f5a service nova] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-f0fb6cc1-ed2a-4588-80b4-f134ee28b855 req-be34bf64-fc01-4aba-892a-1c3458d11f5a service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] No waiting events found dispatching network-vif-plugged-f204cf26-ba59-48a1-b08e-6d8915075961 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:42:10 user nova-compute[71317]: WARNING nova.compute.manager [req-f0fb6cc1-ed2a-4588-80b4-f134ee28b855 req-be34bf64-fc01-4aba-892a-1c3458d11f5a service nova] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Received unexpected event network-vif-plugged-f204cf26-ba59-48a1-b08e-6d8915075961 for instance with vm_state active and task_state deleting.
Apr 14 04:42:11 user nova-compute[71317]: INFO nova.compute.manager [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Took 0.92 seconds to detach 1 volumes for instance.
Apr 14 04:42:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:11 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:42:11 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:42:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.165s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:11 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Deleted allocations for instance 49c7ae26-8d9e-45ac-a189-3ee84e015b27
Apr 14 04:42:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2869c539-e258-45a2-85fb-cf3943c45967 tempest-TestVolumeBootPattern-1780160917 tempest-TestVolumeBootPattern-1780160917-project-member] Lock "49c7ae26-8d9e-45ac-a189-3ee84e015b27" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.137s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Acquiring lock "d3f489a4-972c-4c96-9235-596c23910a9b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "d3f489a4-972c-4c96-9235-596c23910a9b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:42:21 user nova-compute[71317]: INFO nova.compute.claims [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Claim successful on node user
Apr 14 04:42:21 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.216s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:42:21 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:42:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:42:21 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Creating image(s)
Apr 14 04:42:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Acquiring lock "/opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "/opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "/opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG nova.policy [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e8c753ed91f748b8842b022e65b1cfac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3fb95f5c644e4da8b9ab39f07ea79955', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:42:21 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.139s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk 1073741824" returned: 0 in 0.049s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.195s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Checking if we can resize image /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Cannot resize image /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lazy-loading 'migration_context' on Instance uuid d3f489a4-972c-4c96-9235-596c23910a9b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Ensure instance console log exists: /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:22 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Successfully created port: e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Successfully updated port: e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Acquiring lock "refresh_cache-d3f489a4-972c-4c96-9235-596c23910a9b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Acquired lock "refresh_cache-d3f489a4-972c-4c96-9235-596c23910a9b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-62a4bee7-51f6-45c0-87c9-d14e150b6fc8 req-ed1c038e-0e33-4c92-830c-15372d851a5e service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Received event network-changed-e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.compute.manager [req-62a4bee7-51f6-45c0-87c9-d14e150b6fc8 req-ed1c038e-0e33-4c92-830c-15372d851a5e service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Refreshing instance network info cache due to event network-changed-e40f091d-175e-40ad-9b0f-56d648135fff. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-62a4bee7-51f6-45c0-87c9-d14e150b6fc8 req-ed1c038e-0e33-4c92-830c-15372d851a5e service nova] Acquiring lock "refresh_cache-d3f489a4-972c-4c96-9235-596c23910a9b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Updating instance_info_cache with network_info: [{"id": "e40f091d-175e-40ad-9b0f-56d648135fff", "address": "fa:16:3e:c3:8f:62", "network": {"id": "b3ff401b-8246-4be0-b449-be9d89314f53", "bridge": "br-int", "label": "tempest-VolumesActionsTest-87955352-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3fb95f5c644e4da8b9ab39f07ea79955", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape40f091d-17", "ovs_interfaceid": "e40f091d-175e-40ad-9b0f-56d648135fff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Releasing lock "refresh_cache-d3f489a4-972c-4c96-9235-596c23910a9b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Instance network_info: |[{"id": "e40f091d-175e-40ad-9b0f-56d648135fff", "address": "fa:16:3e:c3:8f:62", "network": {"id": "b3ff401b-8246-4be0-b449-be9d89314f53", "bridge": "br-int", "label": "tempest-VolumesActionsTest-87955352-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3fb95f5c644e4da8b9ab39f07ea79955", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape40f091d-17", "ovs_interfaceid": "e40f091d-175e-40ad-9b0f-56d648135fff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-62a4bee7-51f6-45c0-87c9-d14e150b6fc8 req-ed1c038e-0e33-4c92-830c-15372d851a5e service nova] Acquired lock "refresh_cache-d3f489a4-972c-4c96-9235-596c23910a9b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.network.neutron [req-62a4bee7-51f6-45c0-87c9-d14e150b6fc8 req-ed1c038e-0e33-4c92-830c-15372d851a5e service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Refreshing network info cache for port e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Start _get_guest_xml network_info=[{"id": "e40f091d-175e-40ad-9b0f-56d648135fff", "address": "fa:16:3e:c3:8f:62", "network": {"id": "b3ff401b-8246-4be0-b449-be9d89314f53", "bridge": "br-int", "label": "tempest-VolumesActionsTest-87955352-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3fb95f5c644e4da8b9ab39f07ea79955", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape40f091d-17", "ovs_interfaceid": "e40f091d-175e-40ad-9b0f-56d648135fff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:42:23 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:42:23 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:42:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-1029679327',display_name='tempest-VolumesActionsTest-instance-1029679327',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-1029679327',id=45,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3fb95f5c644e4da8b9ab39f07ea79955',ramdisk_id='',reservation_id='r-8hz4hxho',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-318780639',owner_user_name='tempest-VolumesActionsTest-318780639-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:42:22Z,user_data=None,user_id='e8c753ed91f748b8842b022e65b1cfac',uuid=d3f489a4-972c-4c96-9235-596c23910a9b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e40f091d-175e-40ad-9b0f-56d648135fff", "address": "fa:16:3e:c3:8f:62", "network": {"id": "b3ff401b-8246-4be0-b449-be9d89314f53", "bridge": "br-int", "label": "tempest-VolumesActionsTest-87955352-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3fb95f5c644e4da8b9ab39f07ea79955", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape40f091d-17", "ovs_interfaceid": "e40f091d-175e-40ad-9b0f-56d648135fff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Converting VIF {"id": "e40f091d-175e-40ad-9b0f-56d648135fff", "address": "fa:16:3e:c3:8f:62", "network": {"id": "b3ff401b-8246-4be0-b449-be9d89314f53", "bridge": "br-int", "label": "tempest-VolumesActionsTest-87955352-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3fb95f5c644e4da8b9ab39f07ea79955", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape40f091d-17", "ovs_interfaceid": "e40f091d-175e-40ad-9b0f-56d648135fff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c3:8f:62,bridge_name='br-int',has_traffic_filtering=True,id=e40f091d-175e-40ad-9b0f-56d648135fff,network=Network(b3ff401b-8246-4be0-b449-be9d89314f53),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape40f091d-17') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lazy-loading 'pci_devices' on Instance uuid d3f489a4-972c-4c96-9235-596c23910a9b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:42:23 user nova-compute[71317]:   <uuid>d3f489a4-972c-4c96-9235-596c23910a9b</uuid>
Apr 14 04:42:23 user nova-compute[71317]:   <name>instance-0000002d</name>
Apr 14 04:42:23 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:42:23 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:42:23 user nova-compute[71317]:   <metadata>
Apr 14 04:42:23 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:42:23 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:42:23 user nova-compute[71317]:       <nova:name>tempest-VolumesActionsTest-instance-1029679327</nova:name>
Apr 14 04:42:23 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:42:23</nova:creationTime>
Apr 14 04:42:23 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:42:23 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:42:23 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:42:23 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:42:23 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:42:23 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:42:23 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:42:23 user nova-compute[71317]:       <nova:owner>
Apr 14 04:42:23 user nova-compute[71317]:         <nova:user uuid="e8c753ed91f748b8842b022e65b1cfac">tempest-VolumesActionsTest-318780639-project-member</nova:user>
Apr 14 04:42:23 user nova-compute[71317]:         <nova:project uuid="3fb95f5c644e4da8b9ab39f07ea79955">tempest-VolumesActionsTest-318780639</nova:project>
Apr 14 04:42:23 user nova-compute[71317]:       </nova:owner>
Apr 14 04:42:23 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:42:23 user nova-compute[71317]:       <nova:ports>
Apr 14 04:42:23 user nova-compute[71317]:         <nova:port uuid="e40f091d-175e-40ad-9b0f-56d648135fff">
Apr 14 04:42:23 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 14 04:42:23 user nova-compute[71317]:         </nova:port>
Apr 14 04:42:23 user nova-compute[71317]:       </nova:ports>
Apr 14 04:42:23 user nova-compute[71317]:     </nova:instance>
Apr 14 04:42:23 user nova-compute[71317]:   </metadata>
Apr 14 04:42:23 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:42:23 user nova-compute[71317]:     <system>
Apr 14 04:42:23 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:42:23 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:42:23 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:42:23 user nova-compute[71317]:       <entry name="serial">d3f489a4-972c-4c96-9235-596c23910a9b</entry>
Apr 14 04:42:23 user nova-compute[71317]:       <entry name="uuid">d3f489a4-972c-4c96-9235-596c23910a9b</entry>
Apr 14 04:42:23 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:42:23 user nova-compute[71317]:     </system>
Apr 14 04:42:23 user nova-compute[71317]:   </sysinfo>
Apr 14 04:42:23 user nova-compute[71317]:   <os>
Apr 14 04:42:23 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:42:23 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:42:23 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:42:23 user nova-compute[71317]:   </os>
Apr 14 04:42:23 user nova-compute[71317]:   <features>
Apr 14 04:42:23 user nova-compute[71317]:     <acpi/>
Apr 14 04:42:23 user nova-compute[71317]:     <apic/>
Apr 14 04:42:23 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:42:23 user nova-compute[71317]:   </features>
Apr 14 04:42:23 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:42:23 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:42:23 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:42:23 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:42:23 user nova-compute[71317]:   </clock>
Apr 14 04:42:23 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:42:23 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:42:23 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:42:23 user nova-compute[71317]:   </cpu>
Apr 14 04:42:23 user nova-compute[71317]:   <devices>
Apr 14 04:42:23 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:42:23 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:42:23 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk"/>
Apr 14 04:42:23 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:42:23 user nova-compute[71317]:     </disk>
Apr 14 04:42:23 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:42:23 user nova-compute[71317]:       <mac address="fa:16:3e:c3:8f:62"/>
Apr 14 04:42:23 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:42:23 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:42:23 user nova-compute[71317]:       <target dev="tape40f091d-17"/>
Apr 14 04:42:23 user nova-compute[71317]:     </interface>
Apr 14 04:42:23 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:42:23 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/console.log" append="off"/>
Apr 14 04:42:23 user nova-compute[71317]:     </serial>
Apr 14 04:42:23 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:42:23 user nova-compute[71317]:     <video>
Apr 14 04:42:23 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:42:23 user nova-compute[71317]:     </video>
Apr 14 04:42:23 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:42:23 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:42:23 user nova-compute[71317]:     </rng>
Apr 14 04:42:23 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:42:23 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:42:23 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:42:23 user nova-compute[71317]:     </memballoon>
Apr 14 04:42:23 user nova-compute[71317]:   </devices>
Apr 14 04:42:23 user nova-compute[71317]: </domain>
Apr 14 04:42:23 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:42:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-1029679327',display_name='tempest-VolumesActionsTest-instance-1029679327',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-1029679327',id=45,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3fb95f5c644e4da8b9ab39f07ea79955',ramdisk_id='',reservation_id='r-8hz4hxho',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-318780639',owner_user_name='tempest-VolumesActionsTest-318780639-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:42:22Z,user_data=None,user_id='e8c753ed91f748b8842b022e65b1cfac',uuid=d3f489a4-972c-4c96-9235-596c23910a9b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e40f091d-175e-40ad-9b0f-56d648135fff", "address": "fa:16:3e:c3:8f:62", "network": {"id": "b3ff401b-8246-4be0-b449-be9d89314f53", "bridge": "br-int", "label": "tempest-VolumesActionsTest-87955352-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3fb95f5c644e4da8b9ab39f07ea79955", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape40f091d-17", "ovs_interfaceid": "e40f091d-175e-40ad-9b0f-56d648135fff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Converting VIF {"id": "e40f091d-175e-40ad-9b0f-56d648135fff", "address": "fa:16:3e:c3:8f:62", "network": {"id": "b3ff401b-8246-4be0-b449-be9d89314f53", "bridge": "br-int", "label": "tempest-VolumesActionsTest-87955352-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3fb95f5c644e4da8b9ab39f07ea79955", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape40f091d-17", "ovs_interfaceid": "e40f091d-175e-40ad-9b0f-56d648135fff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c3:8f:62,bridge_name='br-int',has_traffic_filtering=True,id=e40f091d-175e-40ad-9b0f-56d648135fff,network=Network(b3ff401b-8246-4be0-b449-be9d89314f53),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape40f091d-17') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG os_vif [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c3:8f:62,bridge_name='br-int',has_traffic_filtering=True,id=e40f091d-175e-40ad-9b0f-56d648135fff,network=Network(b3ff401b-8246-4be0-b449-be9d89314f53),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape40f091d-17') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape40f091d-17, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape40f091d-17, col_values=(('external_ids', {'iface-id': 'e40f091d-175e-40ad-9b0f-56d648135fff', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:c3:8f:62', 'vm-uuid': 'd3f489a4-972c-4c96-9235-596c23910a9b'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:42:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:23 user nova-compute[71317]: INFO os_vif [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c3:8f:62,bridge_name='br-int',has_traffic_filtering=True,id=e40f091d-175e-40ad-9b0f-56d648135fff,network=Network(b3ff401b-8246-4be0-b449-be9d89314f53),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape40f091d-17')
Apr 14 04:42:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:42:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] No VIF found with MAC fa:16:3e:c3:8f:62, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:42:24 user nova-compute[71317]: DEBUG nova.network.neutron [req-62a4bee7-51f6-45c0-87c9-d14e150b6fc8 req-ed1c038e-0e33-4c92-830c-15372d851a5e service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Updated VIF entry in instance network info cache for port e40f091d-175e-40ad-9b0f-56d648135fff. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:42:24 user nova-compute[71317]: DEBUG nova.network.neutron [req-62a4bee7-51f6-45c0-87c9-d14e150b6fc8 req-ed1c038e-0e33-4c92-830c-15372d851a5e service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Updating instance_info_cache with network_info: [{"id": "e40f091d-175e-40ad-9b0f-56d648135fff", "address": "fa:16:3e:c3:8f:62", "network": {"id": "b3ff401b-8246-4be0-b449-be9d89314f53", "bridge": "br-int", "label": "tempest-VolumesActionsTest-87955352-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3fb95f5c644e4da8b9ab39f07ea79955", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape40f091d-17", "ovs_interfaceid": "e40f091d-175e-40ad-9b0f-56d648135fff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:42:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-62a4bee7-51f6-45c0-87c9-d14e150b6fc8 req-ed1c038e-0e33-4c92-830c-15372d851a5e service nova] Releasing lock "refresh_cache-d3f489a4-972c-4c96-9235-596c23910a9b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:42:24 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447329.5374308, 49c7ae26-8d9e-45ac-a189-3ee84e015b27 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:42:24 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] VM Stopped (Lifecycle Event)
Apr 14 04:42:24 user nova-compute[71317]: DEBUG nova.compute.manager [None req-046d2f16-84a4-4ab8-8a5b-8db4056fd184 None None] [instance: 49c7ae26-8d9e-45ac-a189-3ee84e015b27] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:42:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-1ebdf6b6-51e3-4927-ad40-977a4b536367 req-930bfe25-7e9b-4839-9bea-9fa2f952a5e6 service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Received event network-vif-plugged-e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:42:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1ebdf6b6-51e3-4927-ad40-977a4b536367 req-930bfe25-7e9b-4839-9bea-9fa2f952a5e6 service nova] Acquiring lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1ebdf6b6-51e3-4927-ad40-977a4b536367 req-930bfe25-7e9b-4839-9bea-9fa2f952a5e6 service nova] Lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:25 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1ebdf6b6-51e3-4927-ad40-977a4b536367 req-930bfe25-7e9b-4839-9bea-9fa2f952a5e6 service nova] Lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:25 user nova-compute[71317]: DEBUG nova.compute.manager [req-1ebdf6b6-51e3-4927-ad40-977a4b536367 req-930bfe25-7e9b-4839-9bea-9fa2f952a5e6 service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] No waiting events found dispatching network-vif-plugged-e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:42:25 user nova-compute[71317]: WARNING nova.compute.manager [req-1ebdf6b6-51e3-4927-ad40-977a4b536367 req-930bfe25-7e9b-4839-9bea-9fa2f952a5e6 service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Received unexpected event network-vif-plugged-e40f091d-175e-40ad-9b0f-56d648135fff for instance with vm_state building and task_state spawning.
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447347.3797505, d3f489a4-972c-4c96-9235-596c23910a9b => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:42:27 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] VM Resumed (Lifecycle Event)
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:42:27 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Instance spawned successfully.
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:42:27 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447347.385263, d3f489a4-972c-4c96-9235-596c23910a9b => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:42:27 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] VM Started (Lifecycle Event)
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:42:27 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:42:27 user nova-compute[71317]: INFO nova.compute.manager [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Took 5.56 seconds to spawn the instance on the hypervisor.
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:42:27 user nova-compute[71317]: INFO nova.compute.manager [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Took 6.08 seconds to build instance.
Apr 14 04:42:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ea77dae-7aed-4622-b0b2-d3e4aaae9a11 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "d3f489a4-972c-4c96-9235-596c23910a9b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.167s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-18b71155-06ff-4236-9116-feee1b1dc369 req-1d315775-ce6c-4e67-9f24-dbec4256f40d service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Received event network-vif-plugged-e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-18b71155-06ff-4236-9116-feee1b1dc369 req-1d315775-ce6c-4e67-9f24-dbec4256f40d service nova] Acquiring lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-18b71155-06ff-4236-9116-feee1b1dc369 req-1d315775-ce6c-4e67-9f24-dbec4256f40d service nova] Lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-18b71155-06ff-4236-9116-feee1b1dc369 req-1d315775-ce6c-4e67-9f24-dbec4256f40d service nova] Lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:27 user nova-compute[71317]: DEBUG nova.compute.manager [req-18b71155-06ff-4236-9116-feee1b1dc369 req-1d315775-ce6c-4e67-9f24-dbec4256f40d service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] No waiting events found dispatching network-vif-plugged-e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:42:27 user nova-compute[71317]: WARNING nova.compute.manager [req-18b71155-06ff-4236-9116-feee1b1dc369 req-1d315775-ce6c-4e67-9f24-dbec4256f40d service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Received unexpected event network-vif-plugged-e40f091d-175e-40ad-9b0f-56d648135fff for instance with vm_state active and task_state None.
Apr 14 04:42:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:38 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:44 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:44 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:42:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:42:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:42:44 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:42:44 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Updating instance_info_cache with network_info: [{"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:42:44 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:42:44 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:42:44 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:45 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:46 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:48 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 04:42:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] There are 0 instances to clean {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:42:53 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:42:53 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:42:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8445MB free_disk=26.304309844970703GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3d8009dc-c463-41f2-8b36-21862b7ad5ca actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 77f6bb83-54ab-485e-8943-742f2f576607 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance d3f489a4-972c-4c96-9235-596c23910a9b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing inventories for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating ProviderTree inventory for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating inventory in ProviderTree for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing aggregate associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, aggregates: None {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing trait associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, traits: COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSE41,HW_CPU_X86_SSE42,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_TRUSTED_CERTS,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_ACCELERATORS,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NET_VIF_MODEL_E1000,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_NODE,HW_CPU_X86_SSE2,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,HW_CPU_X86_SSSE3,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_STORAGE_BUS_USB,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_NE2K_PCI {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 04:42:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:54 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:42:54 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:42:54 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:42:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.544s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:42:55 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:55 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:55 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:42:58 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:42:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances with incomplete migration  {{(pid=71317) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 04:42:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:42:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-b202f8dd-c123-44b8-9d1a-425043d28a91 req-47e10e43-0796-47d6-8ef8-413a737d0fee service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Received event network-changed-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:43:07 user nova-compute[71317]: DEBUG nova.compute.manager [req-b202f8dd-c123-44b8-9d1a-425043d28a91 req-47e10e43-0796-47d6-8ef8-413a737d0fee service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Refreshing instance network info cache due to event network-changed-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:43:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b202f8dd-c123-44b8-9d1a-425043d28a91 req-47e10e43-0796-47d6-8ef8-413a737d0fee service nova] Acquiring lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:43:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b202f8dd-c123-44b8-9d1a-425043d28a91 req-47e10e43-0796-47d6-8ef8-413a737d0fee service nova] Acquired lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:43:07 user nova-compute[71317]: DEBUG nova.network.neutron [req-b202f8dd-c123-44b8-9d1a-425043d28a91 req-47e10e43-0796-47d6-8ef8-413a737d0fee service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Refreshing network info cache for port 6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:43:08 user nova-compute[71317]: DEBUG nova.network.neutron [req-b202f8dd-c123-44b8-9d1a-425043d28a91 req-47e10e43-0796-47d6-8ef8-413a737d0fee service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Updated VIF entry in instance network info cache for port 6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:43:08 user nova-compute[71317]: DEBUG nova.network.neutron [req-b202f8dd-c123-44b8-9d1a-425043d28a91 req-47e10e43-0796-47d6-8ef8-413a737d0fee service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Updating instance_info_cache with network_info: [{"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.109", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:43:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b202f8dd-c123-44b8-9d1a-425043d28a91 req-47e10e43-0796-47d6-8ef8-413a737d0fee service nova] Releasing lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:43:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG nova.objects.instance [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'flavor' on Instance uuid 3d8009dc-c463-41f2-8b36-21862b7ad5ca {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.055s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:43:10 user nova-compute[71317]: INFO nova.compute.manager [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Attaching volume 5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 to /dev/vdb
Apr 14 04:43:10 user nova-compute[71317]: DEBUG os_brick.utils [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1030443b-5e52-421e-b9df-828b26afa474]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[060751a8-b9f4-456b-a117-3b96e02b30cd]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[de1c7752-8702-4d9d-87ce-5f2aeb6c8689]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee433a02-07fd-4d2c-a500-75c3a5682045]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:43:10 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:43:10 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a77eaf7d-2b60-4680-b7db-a1c662095348]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG os_brick.utils [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] <== get_connector_properties: return (133ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:43:10 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Updating existing volume attachment record: 6eb9607f-5b87-4c3f-bc69-807b73c894e7 {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:43:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ULBNyXqg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:43:14 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:43:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:43:14 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:43:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:43:14 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d3940c9-acc9-4f83-be75-6194385c066f]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:43:14 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5ed4c1e-2065-4e80-976a-d12793a3f1cf]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7] added
Apr 14 04:43:14 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e4da92f-b307-4d62-abff-96fb317140df]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5defcac7-8483-4cd6-849f-54ca35c59244]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:43:14 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ULBNyXqg {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ULBNyXqg" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[299b5e0c-4eeb-4b4b-9ba4-c1bdd3207e2d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'ULBNyXqg'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3332c457-962d-465c-9543-b35aaf35df89]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f93c564-16f7-46fc-b2ad-be8c83138c97]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 14 04:43:15 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 14 04:43:15 user nova-compute[71317]:  {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:43:15 user nova-compute[71317]: WARNING os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 14 04:43:15 user nova-compute[71317]: 
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --login" returned: 0 in 0.046s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[68f7ba54-5a90-4ae4-a8eb-d29d02e4be18]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7, portal: 172.16.0.220,3260]
Apr 14 04:43:15 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7, portal: 172.16.0.220,3260] successful.
Apr 14 04:43:15 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[41b10469-a6c6-4c26-b66b-ae56ccac767f]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[35dcdfe5-3f75-490f-8820-b83c2187489a]: (4, ('tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 (non-flash)
Apr 14 04:43:15 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsi session list stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 (non-flash)
Apr 14 04:43:15 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.256s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] HCTL ('33', '-', '-', 0) found on session 52 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[22f2058c-a3d4-4043-b92b-01dd99a41db9]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Searching for a device in session 52 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:43:15 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ULBNyXqg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:43:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.288s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:43:16 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] <== connect_volume: return (1291ms) {'type': 'block', 'scsi_wwn': '26639376132353630', 'path': '/dev/sdb'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:43:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26639376132353630', 'path': '/dev/sdb'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:43:16 user nova-compute[71317]: DEBUG nova.objects.instance [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'flavor' on Instance uuid 3d8009dc-c463-41f2-8b36-21862b7ad5ca {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:43:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:43:16 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:43:16 user nova-compute[71317]:   <source dev="/dev/sdb"/>
Apr 14 04:43:16 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:43:16 user nova-compute[71317]:   <serial>5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7</serial>
Apr 14 04:43:16 user nova-compute[71317]: </disk>
Apr 14 04:43:16 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:43:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:43:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:43:16 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No VIF found with MAC fa:16:3e:1b:b5:53, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:43:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4c817f47-2d17-4cfa-beb0-9f6173f158cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.407s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:43:18 user nova-compute[71317]: INFO nova.compute.manager [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Detaching volume 5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7
Apr 14 04:43:18 user nova-compute[71317]: INFO nova.virt.block_device [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Attempting to driver detach volume 5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 from mountpoint /dev/vdb
Apr 14 04:43:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Attempting to detach device vdb from instance 3d8009dc-c463-41f2-8b36-21862b7ad5ca from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:43:18 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:43:18 user nova-compute[71317]:   <source dev="/dev/sdb"/>
Apr 14 04:43:18 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:43:18 user nova-compute[71317]:   <serial>5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7</serial>
Apr 14 04:43:18 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:43:18 user nova-compute[71317]: </disk>
Apr 14 04:43:18 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:43:18 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully detached device vdb from instance 3d8009dc-c463-41f2-8b36-21862b7ad5ca from the persistent domain config.
Apr 14 04:43:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 3d8009dc-c463-41f2-8b36-21862b7ad5ca from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:43:18 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:43:18 user nova-compute[71317]:   <source dev="/dev/sdb"/>
Apr 14 04:43:18 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:43:18 user nova-compute[71317]:   <serial>5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7</serial>
Apr 14 04:43:18 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:43:18 user nova-compute[71317]: </disk>
Apr 14 04:43:18 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681447398.7129588, 3d8009dc-c463-41f2-8b36-21862b7ad5ca => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 3d8009dc-c463-41f2-8b36-21862b7ad5ca {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:43:18 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully detached device vdb from instance 3d8009dc-c463-41f2-8b36-21862b7ad5ca from the live domain config.
Apr 14 04:43:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ULBNyXqg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:43:18 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d3f2a68-eab8-4419-9bbb-2314c3658a40]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:43:18 user nova-compute[71317]: No targets found.
Apr 14 04:43:18 user nova-compute[71317]: iSNS:
Apr 14 04:43:18 user nova-compute[71317]: No targets found.
Apr 14 04:43:18 user nova-compute[71317]: STATIC:
Apr 14 04:43:18 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7
Apr 14 04:43:18 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:43:18 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:43:18 user nova-compute[71317]: FIRMWARE:
Apr 14 04:43:18 user nova-compute[71317]: No targets found.
Apr 14 04:43:18 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:43:18 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:43:18 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1b8d3342-9085-4e26-b8d3-e8882fe92d9e]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5999e7c7-a5ed-4c9b-997a-656a81ae8439]: (4, ('tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 (non-flash)
Apr 14 04:43:18 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsi session list stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 (non-flash)
Apr 14 04:43:18 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f84b4136e60>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7'): ({'sdb'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Removing single pathed devices sdb {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7e0f5d2-802c-495b-9d95-8bfc3751da15]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.102s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[131fed98-fb1d-4651-819c-dd33ba9828a0]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Flushing IO for device /dev/sdb {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac796806-8262-4c1c-ad0b-ebde5c93ac63]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.048s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[4cc0bcfc-41f2-4aa6-bd42-1c9e94fd0cf2]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:43:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca12d759-170a-4f15-ac64-2248c4d9ff8d]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --logout" returned: 0 in 0.048s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bd1a9db-8faa-40ca-a661-45b511c10540]: (4, ('Logging out of session [sid: 52, target: iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7, portal: 172.16.0.220,3260]\nLogout of [sid: 52, target: iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 52, target: iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7, portal: 172.16.0.220,3260]
Apr 14 04:43:19 user nova-compute[71317]: Logout of [sid: 52, target: iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7, portal: 172.16.0.220,3260] successful.
Apr 14 04:43:19 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b1bb6a7-c9a7-4ac1-9a24-c912f5f812e7 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ef85eb8-0ecf-41d0-8d14-70a06e675cd1]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.357s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] <== disconnect_volume: return (359ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG nova.objects.instance [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'flavor' on Instance uuid 3d8009dc-c463-41f2-8b36-21862b7ad5ca {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:43:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-3f7d9c3c-b0c9-4932-8a90-cba7766d76cf tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.610s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:43:23 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_running_deleted_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:43:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_power_states {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 3d8009dc-c463-41f2-8b36-21862b7ad5ca {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid d3f489a4-972c-4c96-9235-596c23910a9b {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "77f6bb83-54ab-485e-8943-742f2f576607" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "d3f489a4-972c-4c96-9235-596c23910a9b" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "d3f489a4-972c-4c96-9235-596c23910a9b" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.026s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "77f6bb83-54ab-485e-8943-742f2f576607" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.029s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:43:31 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "d3f489a4-972c-4c96-9235-596c23910a9b" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.032s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:43:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:43:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:39 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:43:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:45 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:43:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:43:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:43:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:43:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:43:45 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:43:45 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lazy-loading 'info_cache' on Instance uuid 3d8009dc-c463-41f2-8b36-21862b7ad5ca {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:43:45 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Updating instance_info_cache with network_info: [{"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.109", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:43:45 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-3d8009dc-c463-41f2-8b36-21862b7ad5ca" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:43:45 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:43:45 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:43:46 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:43:48 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:43:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:43:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:43:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:43:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:43:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:50 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-1b90a958-7688-428e-8a65-15de6512bb2d req-9f1ba33d-3a0d-4479-a380-d641c8393b41 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-changed-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG nova.compute.manager [req-1b90a958-7688-428e-8a65-15de6512bb2d req-9f1ba33d-3a0d-4479-a380-d641c8393b41 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Refreshing instance network info cache due to event network-changed-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1b90a958-7688-428e-8a65-15de6512bb2d req-9f1ba33d-3a0d-4479-a380-d641c8393b41 service nova] Acquiring lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1b90a958-7688-428e-8a65-15de6512bb2d req-9f1ba33d-3a0d-4479-a380-d641c8393b41 service nova] Acquired lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG nova.network.neutron [req-1b90a958-7688-428e-8a65-15de6512bb2d req-9f1ba33d-3a0d-4479-a380-d641c8393b41 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Refreshing network info cache for port 440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG nova.network.neutron [req-1b90a958-7688-428e-8a65-15de6512bb2d req-9f1ba33d-3a0d-4479-a380-d641c8393b41 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updated VIF entry in instance network info cache for port 440e7ef0-4f4c-4923-ab57-8e1ba56dbea3. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG nova.network.neutron [req-1b90a958-7688-428e-8a65-15de6512bb2d req-9f1ba33d-3a0d-4479-a380-d641c8393b41 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updating instance_info_cache with network_info: [{"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-1b90a958-7688-428e-8a65-15de6512bb2d req-9f1ba33d-3a0d-4479-a380-d641c8393b41 service nova] Releasing lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:52 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:53 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:43:53 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:43:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8532MB free_disk=26.302322387695312GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 3d8009dc-c463-41f2-8b36-21862b7ad5ca actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 77f6bb83-54ab-485e-8943-742f2f576607 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance d3f489a4-972c-4c96-9235-596c23910a9b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.227s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:43:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:43:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "77f6bb83-54ab-485e-8943-742f2f576607" acquired by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:43:55 user nova-compute[71317]: INFO nova.compute.manager [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Shelving
Apr 14 04:43:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:55 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:43:55 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:43:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:43:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-860b7394-6da7-43f9-bd07-8bdf3ebaaa66 req-eb35a7f2-0770-4854-a463-4b648d522206 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-vif-unplugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:43:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-860b7394-6da7-43f9-bd07-8bdf3ebaaa66 req-eb35a7f2-0770-4854-a463-4b648d522206 service nova] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:43:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-860b7394-6da7-43f9-bd07-8bdf3ebaaa66 req-eb35a7f2-0770-4854-a463-4b648d522206 service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:43:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-860b7394-6da7-43f9-bd07-8bdf3ebaaa66 req-eb35a7f2-0770-4854-a463-4b648d522206 service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:43:55 user nova-compute[71317]: DEBUG nova.compute.manager [req-860b7394-6da7-43f9-bd07-8bdf3ebaaa66 req-eb35a7f2-0770-4854-a463-4b648d522206 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] No waiting events found dispatching network-vif-unplugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:43:55 user nova-compute[71317]: WARNING nova.compute.manager [req-860b7394-6da7-43f9-bd07-8bdf3ebaaa66 req-eb35a7f2-0770-4854-a463-4b648d522206 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received unexpected event network-vif-unplugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 for instance with vm_state active and task_state shelving.
Apr 14 04:43:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Instance destroyed successfully.
Apr 14 04:43:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Beginning cold snapshot process
Apr 14 04:43:56 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:43:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk /opt/stack/data/nova/instances/snapshots/tmpxla1l5qh/df5d9a3f84e041ad9efb67240c712f37 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:43:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk /opt/stack/data/nova/instances/snapshots/tmpxla1l5qh/df5d9a3f84e041ad9efb67240c712f37" returned: 0 in 0.410s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:43:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Snapshot extracted, beginning image upload
Apr 14 04:43:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-3eb53b0c-b73b-483d-83bf-780381d0774a req-c0b3641a-b879-41c8-8c2b-27edf1b6e410 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:43:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3eb53b0c-b73b-483d-83bf-780381d0774a req-c0b3641a-b879-41c8-8c2b-27edf1b6e410 service nova] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:43:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3eb53b0c-b73b-483d-83bf-780381d0774a req-c0b3641a-b879-41c8-8c2b-27edf1b6e410 service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:43:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-3eb53b0c-b73b-483d-83bf-780381d0774a req-c0b3641a-b879-41c8-8c2b-27edf1b6e410 service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:43:57 user nova-compute[71317]: DEBUG nova.compute.manager [req-3eb53b0c-b73b-483d-83bf-780381d0774a req-c0b3641a-b879-41c8-8c2b-27edf1b6e410 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] No waiting events found dispatching network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:43:57 user nova-compute[71317]: WARNING nova.compute.manager [req-3eb53b0c-b73b-483d-83bf-780381d0774a req-c0b3641a-b879-41c8-8c2b-27edf1b6e410 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received unexpected event network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 for instance with vm_state active and task_state shelving_image_uploading.
Apr 14 04:43:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:58 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Snapshot image upload complete
Apr 14 04:43:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:43:58 user nova-compute[71317]: INFO nova.compute.manager [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Shelve offloading
Apr 14 04:43:58 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Instance destroyed successfully.
Apr 14 04:43:58 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:43:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:43:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquired lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:43:58 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:43:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:43:59 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updating instance_info_cache with network_info: [{"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:43:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Releasing lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:44:00 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Instance destroyed successfully.
Apr 14 04:44:00 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'resources' on Instance uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:42:00Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1249800375',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1249800375',id=44,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH6fraox0ufDVlTeIxqf3LA1Jqk/6GB3uKQGBsTlrxVUqCfAkV+hrTL0aqpPNJ0Y4OrCsZ9AqcY4PXRY4CmEWIke2qyPR134zIiDdDnr3VeL889NFNXZb2Fua8pApWkvww==',key_name='tempest-keypair-1996810505',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:42:07Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='c10c8cf348e04e44afbaae93a4b60e36',ramdisk_id='',reservation_id='r-4s0bcxm4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2141578557',owner_user_name='tempest-AttachVolumeShelveTestJSON-2141578557-project-member',shelved_at='2023-04-14T04:43:58.799231',shelved_host='user',shelved_image_id='ed0f5b6b-1ab3-461f-987b-1c736a691553'},tags=<?>,task_state='shelving_offloading',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:43:57Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='67d6c699fc524554a6f4983af91d58fc',uuid=77f6bb83-54ab-485e-8943-742f2f576607,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='shelved') vif={"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converting VIF {"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG os_vif [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap440e7ef0-4f, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:44:00 user nova-compute[71317]: INFO os_vif [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f')
Apr 14 04:44:00 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Deleting instance files /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607_del
Apr 14 04:44:00 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Deletion of /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607_del complete
Apr 14 04:44:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-e840d1fd-3eb5-475b-9f4f-d9062a548a66 req-7a3e04d4-f7e3-42ca-b235-4ffb54546820 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-changed-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-e840d1fd-3eb5-475b-9f4f-d9062a548a66 req-7a3e04d4-f7e3-42ca-b235-4ffb54546820 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Refreshing instance network info cache due to event network-changed-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e840d1fd-3eb5-475b-9f4f-d9062a548a66 req-7a3e04d4-f7e3-42ca-b235-4ffb54546820 service nova] Acquiring lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e840d1fd-3eb5-475b-9f4f-d9062a548a66 req-7a3e04d4-f7e3-42ca-b235-4ffb54546820 service nova] Acquired lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG nova.network.neutron [req-e840d1fd-3eb5-475b-9f4f-d9062a548a66 req-7a3e04d4-f7e3-42ca-b235-4ffb54546820 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Refreshing network info cache for port 440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:44:00 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Deleted allocations for instance 77f6bb83-54ab-485e-8943-742f2f576607
Apr 14 04:44:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'numa_topology' on Instance uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.151s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c7b7d9d1-68a5-4de5-8a13-3fd2cc2c6d39 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "77f6bb83-54ab-485e-8943-742f2f576607" "released" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: held 5.108s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG nova.network.neutron [req-e840d1fd-3eb5-475b-9f4f-d9062a548a66 req-7a3e04d4-f7e3-42ca-b235-4ffb54546820 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updated VIF entry in instance network info cache for port 440e7ef0-4f4c-4923-ab57-8e1ba56dbea3. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG nova.network.neutron [req-e840d1fd-3eb5-475b-9f4f-d9062a548a66 req-7a3e04d4-f7e3-42ca-b235-4ffb54546820 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updating instance_info_cache with network_info: [{"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:44:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-e840d1fd-3eb5-475b-9f4f-d9062a548a66 req-7a3e04d4-f7e3-42ca-b235-4ffb54546820 service nova] Releasing lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:44:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:08 user nova-compute[71317]: INFO nova.compute.manager [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Terminating instance
Apr 14 04:44:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-16c6b243-806a-40cd-825f-7270b9be6658 req-e39f5d14-f087-462c-b981-a22ffb27946e service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Received event network-vif-unplugged-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-16c6b243-806a-40cd-825f-7270b9be6658 req-e39f5d14-f087-462c-b981-a22ffb27946e service nova] Acquiring lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-16c6b243-806a-40cd-825f-7270b9be6658 req-e39f5d14-f087-462c-b981-a22ffb27946e service nova] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-16c6b243-806a-40cd-825f-7270b9be6658 req-e39f5d14-f087-462c-b981-a22ffb27946e service nova] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-16c6b243-806a-40cd-825f-7270b9be6658 req-e39f5d14-f087-462c-b981-a22ffb27946e service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] No waiting events found dispatching network-vif-unplugged-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:44:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-16c6b243-806a-40cd-825f-7270b9be6658 req-e39f5d14-f087-462c-b981-a22ffb27946e service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Received event network-vif-unplugged-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:44:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Instance destroyed successfully.
Apr 14 04:44:09 user nova-compute[71317]: DEBUG nova.objects.instance [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'resources' on Instance uuid 3d8009dc-c463-41f2-8b36-21862b7ad5ca {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:44:09 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:41:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-610195412',display_name='tempest-AttachVolumeNegativeTest-server-610195412',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-610195412',id=43,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC3TphBlHcrFQUWfHI9Rd+a22lD9cX0GeecKjEQEW/ftTRl57yNsm3ZsNJGGAxbXJX4cCwx+4dgcJcdGA6/AzTSvdleEod5ijJD25/l5NF3G7sPYAP/bwHLaLqv+XnXDSQ==',key_name='tempest-keypair-2017831093',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:41:22Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='601e2806907a48d3995cc90b54da3c8c',ramdisk_id='',reservation_id='r-lm0pepyf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-1197484356',owner_user_name='tempest-AttachVolumeNegativeTest-1197484356-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:41:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7cab2461ab0d4eb08b62feaf6396bc87',uuid=3d8009dc-c463-41f2-8b36-21862b7ad5ca,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.109", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:44:09 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converting VIF {"id": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "address": "fa:16:3e:1b:b5:53", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.109", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9b1d34-ea", "ovs_interfaceid": "6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:44:09 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:1b:b5:53,bridge_name='br-int',has_traffic_filtering=True,id=6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9b1d34-ea') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:44:09 user nova-compute[71317]: DEBUG os_vif [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:1b:b5:53,bridge_name='br-int',has_traffic_filtering=True,id=6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9b1d34-ea') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:44:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap6d9b1d34-ea, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:44:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:44:09 user nova-compute[71317]: INFO os_vif [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:1b:b5:53,bridge_name='br-int',has_traffic_filtering=True,id=6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9b1d34-ea')
Apr 14 04:44:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Deleting instance files /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca_del
Apr 14 04:44:09 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Deletion of /opt/stack/data/nova/instances/3d8009dc-c463-41f2-8b36-21862b7ad5ca_del complete
Apr 14 04:44:09 user nova-compute[71317]: INFO nova.compute.manager [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Took 0.82 seconds to destroy the instance on the hypervisor.
Apr 14 04:44:09 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:44:09 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:44:09 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:44:10 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:44:10 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Took 0.79 seconds to deallocate network for instance.
Apr 14 04:44:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-95011e8a-9093-4c90-9672-247fbb9e340a req-e77f16ce-0805-4c4c-8957-cc9085abc149 service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Received event network-vif-deleted-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:44:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:10 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:44:10 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:44:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.140s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:10 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Deleted allocations for instance 3d8009dc-c463-41f2-8b36-21862b7ad5ca
Apr 14 04:44:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-09c6abaa-2621-4b36-8266-1aca7a222a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.929s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-cf4135aa-1ae3-44af-a1a4-e93ca0d97554 req-6f27a626-ee33-4247-88a1-d62759427495 service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Received event network-vif-plugged-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:44:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cf4135aa-1ae3-44af-a1a4-e93ca0d97554 req-6f27a626-ee33-4247-88a1-d62759427495 service nova] Acquiring lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cf4135aa-1ae3-44af-a1a4-e93ca0d97554 req-6f27a626-ee33-4247-88a1-d62759427495 service nova] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-cf4135aa-1ae3-44af-a1a4-e93ca0d97554 req-6f27a626-ee33-4247-88a1-d62759427495 service nova] Lock "3d8009dc-c463-41f2-8b36-21862b7ad5ca-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-cf4135aa-1ae3-44af-a1a4-e93ca0d97554 req-6f27a626-ee33-4247-88a1-d62759427495 service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] No waiting events found dispatching network-vif-plugged-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:44:10 user nova-compute[71317]: WARNING nova.compute.manager [req-cf4135aa-1ae3-44af-a1a4-e93ca0d97554 req-6f27a626-ee33-4247-88a1-d62759427495 service nova] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Received unexpected event network-vif-plugged-6d9b1d34-ea57-4e92-bdc3-1bd292ce5e68 for instance with vm_state deleted and task_state None.
Apr 14 04:44:11 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447436.1381872, 77f6bb83-54ab-485e-8943-742f2f576607 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:44:11 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] VM Stopped (Lifecycle Event)
Apr 14 04:44:11 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3f279794-5a3c-43cc-be81-f4ac3c108961 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:44:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Acquiring lock "d3f489a4-972c-4c96-9235-596c23910a9b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "d3f489a4-972c-4c96-9235-596c23910a9b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Acquiring lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:17 user nova-compute[71317]: INFO nova.compute.manager [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Terminating instance
Apr 14 04:44:17 user nova-compute[71317]: DEBUG nova.compute.manager [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-db65b880-2e79-4858-833a-cad688facf70 req-7cb3fa03-df64-4f4c-8fe2-fecae7d012a0 service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Received event network-vif-unplugged-e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-db65b880-2e79-4858-833a-cad688facf70 req-7cb3fa03-df64-4f4c-8fe2-fecae7d012a0 service nova] Acquiring lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-db65b880-2e79-4858-833a-cad688facf70 req-7cb3fa03-df64-4f4c-8fe2-fecae7d012a0 service nova] Lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-db65b880-2e79-4858-833a-cad688facf70 req-7cb3fa03-df64-4f4c-8fe2-fecae7d012a0 service nova] Lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-db65b880-2e79-4858-833a-cad688facf70 req-7cb3fa03-df64-4f4c-8fe2-fecae7d012a0 service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] No waiting events found dispatching network-vif-unplugged-e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-db65b880-2e79-4858-833a-cad688facf70 req-7cb3fa03-df64-4f4c-8fe2-fecae7d012a0 service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Received event network-vif-unplugged-e40f091d-175e-40ad-9b0f-56d648135fff for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Instance destroyed successfully.
Apr 14 04:44:17 user nova-compute[71317]: DEBUG nova.objects.instance [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lazy-loading 'resources' on Instance uuid d3f489a4-972c-4c96-9235-596c23910a9b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:42:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-1029679327',display_name='tempest-VolumesActionsTest-instance-1029679327',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-1029679327',id=45,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:42:27Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3fb95f5c644e4da8b9ab39f07ea79955',ramdisk_id='',reservation_id='r-8hz4hxho',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-318780639',owner_user_name='tempest-VolumesActionsTest-318780639-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:42:28Z,user_data=None,user_id='e8c753ed91f748b8842b022e65b1cfac',uuid=d3f489a4-972c-4c96-9235-596c23910a9b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e40f091d-175e-40ad-9b0f-56d648135fff", "address": "fa:16:3e:c3:8f:62", "network": {"id": "b3ff401b-8246-4be0-b449-be9d89314f53", "bridge": "br-int", "label": "tempest-VolumesActionsTest-87955352-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3fb95f5c644e4da8b9ab39f07ea79955", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape40f091d-17", "ovs_interfaceid": "e40f091d-175e-40ad-9b0f-56d648135fff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Converting VIF {"id": "e40f091d-175e-40ad-9b0f-56d648135fff", "address": "fa:16:3e:c3:8f:62", "network": {"id": "b3ff401b-8246-4be0-b449-be9d89314f53", "bridge": "br-int", "label": "tempest-VolumesActionsTest-87955352-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3fb95f5c644e4da8b9ab39f07ea79955", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape40f091d-17", "ovs_interfaceid": "e40f091d-175e-40ad-9b0f-56d648135fff", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c3:8f:62,bridge_name='br-int',has_traffic_filtering=True,id=e40f091d-175e-40ad-9b0f-56d648135fff,network=Network(b3ff401b-8246-4be0-b449-be9d89314f53),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape40f091d-17') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG os_vif [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c3:8f:62,bridge_name='br-int',has_traffic_filtering=True,id=e40f091d-175e-40ad-9b0f-56d648135fff,network=Network(b3ff401b-8246-4be0-b449-be9d89314f53),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape40f091d-17') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape40f091d-17, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:44:17 user nova-compute[71317]: INFO os_vif [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c3:8f:62,bridge_name='br-int',has_traffic_filtering=True,id=e40f091d-175e-40ad-9b0f-56d648135fff,network=Network(b3ff401b-8246-4be0-b449-be9d89314f53),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape40f091d-17')
Apr 14 04:44:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Deleting instance files /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b_del
Apr 14 04:44:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Deletion of /opt/stack/data/nova/instances/d3f489a4-972c-4c96-9235-596c23910a9b_del complete
Apr 14 04:44:17 user nova-compute[71317]: INFO nova.compute.manager [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Took 0.65 seconds to destroy the instance on the hypervisor.
Apr 14 04:44:17 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:44:17 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:44:18 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:44:18 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Took 0.45 seconds to deallocate network for instance.
Apr 14 04:44:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-2fd1b711-3aa4-4253-8fb6-dd72795988bd req-8313ffd4-3c65-4fc0-8d55-b5007e19091c service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Received event network-vif-deleted-e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:44:18 user nova-compute[71317]: INFO nova.compute.manager [req-2fd1b711-3aa4-4253-8fb6-dd72795988bd req-8313ffd4-3c65-4fc0-8d55-b5007e19091c service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Neutron deleted interface e40f091d-175e-40ad-9b0f-56d648135fff; detaching it from the instance and deleting it from the info cache
Apr 14 04:44:18 user nova-compute[71317]: DEBUG nova.network.neutron [req-2fd1b711-3aa4-4253-8fb6-dd72795988bd req-8313ffd4-3c65-4fc0-8d55-b5007e19091c service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:44:18 user nova-compute[71317]: DEBUG nova.compute.manager [req-2fd1b711-3aa4-4253-8fb6-dd72795988bd req-8313ffd4-3c65-4fc0-8d55-b5007e19091c service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Detach interface failed, port_id=e40f091d-175e-40ad-9b0f-56d648135fff, reason: Instance d3f489a4-972c-4c96-9235-596c23910a9b could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:44:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:18 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:44:18 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:44:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.101s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:18 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Deleted allocations for instance d3f489a4-972c-4c96-9235-596c23910a9b
Apr 14 04:44:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-906527f4-0fe2-458c-a8eb-c7f499112fa2 tempest-VolumesActionsTest-318780639 tempest-VolumesActionsTest-318780639-project-member] Lock "d3f489a4-972c-4c96-9235-596c23910a9b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.363s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-2d8adfb1-8267-40f3-85e1-4ab1d824f646 req-89c2bab7-15e8-4330-a5ec-6aeb6e0b1a77 service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Received event network-vif-plugged-e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:44:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2d8adfb1-8267-40f3-85e1-4ab1d824f646 req-89c2bab7-15e8-4330-a5ec-6aeb6e0b1a77 service nova] Acquiring lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2d8adfb1-8267-40f3-85e1-4ab1d824f646 req-89c2bab7-15e8-4330-a5ec-6aeb6e0b1a77 service nova] Lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-2d8adfb1-8267-40f3-85e1-4ab1d824f646 req-89c2bab7-15e8-4330-a5ec-6aeb6e0b1a77 service nova] Lock "d3f489a4-972c-4c96-9235-596c23910a9b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-2d8adfb1-8267-40f3-85e1-4ab1d824f646 req-89c2bab7-15e8-4330-a5ec-6aeb6e0b1a77 service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] No waiting events found dispatching network-vif-plugged-e40f091d-175e-40ad-9b0f-56d648135fff {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:44:19 user nova-compute[71317]: WARNING nova.compute.manager [req-2d8adfb1-8267-40f3-85e1-4ab1d824f646 req-89c2bab7-15e8-4330-a5ec-6aeb6e0b1a77 service nova] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Received unexpected event network-vif-plugged-e40f091d-175e-40ad-9b0f-56d648135fff for instance with vm_state deleted and task_state None.
Apr 14 04:44:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:24 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447449.3328516, 3d8009dc-c463-41f2-8b36-21862b7ad5ca => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:44:24 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] VM Stopped (Lifecycle Event)
Apr 14 04:44:24 user nova-compute[71317]: DEBUG nova.compute.manager [None req-23c935ca-97c7-479f-af14-a34e0a22aa06 None None] [instance: 3d8009dc-c463-41f2-8b36-21862b7ad5ca] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:44:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:32 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447457.7589023, d3f489a4-972c-4c96-9235-596c23910a9b => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:44:32 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] VM Stopped (Lifecycle Event)
Apr 14 04:44:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-3f520d3c-9cc4-417f-b4d6-7f1674327033 None None] [instance: d3f489a4-972c-4c96-9235-596c23910a9b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:44:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:44:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:44:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:44:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:44:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:44:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:44:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:44:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:44:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:40 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:44:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:44:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:44:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:44:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:44:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:44:46 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:44:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:44:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Skipping network cache update for instance because it has been migrated to another host. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9836}}
Apr 14 04:44:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Didn't find any instances for network info cache update. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 04:44:47 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:44:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:48 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:44:50 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:44:51 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:44:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:44:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:44:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:44:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:44:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:44:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:44:54 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:44:54 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:44:54 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8783MB free_disk=26.322975158691406GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:44:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.151s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:44:56 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:44:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:45:02 user nova-compute[71317]: INFO nova.compute.claims [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Claim successful on node user
Apr 14 04:45:02 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.179s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG nova.network.neutron [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:45:02 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:45:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:45:02 user nova-compute[71317]: DEBUG nova.policy [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7cab2461ab0d4eb08b62feaf6396bc87', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '601e2806907a48d3995cc90b54da3c8c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:45:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Creating image(s)
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "/opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "/opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "/opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.130s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk 1073741824" returned: 0 in 0.048s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.190s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Cannot resize image /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG nova.objects.instance [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'migration_context' on Instance uuid f5a130f4-02c3-41fc-bbdb-d2b844e1320b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG nova.network.neutron [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Successfully created port: 7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Ensure instance console log exists: /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.network.neutron [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Successfully updated port: 7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquired lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.network.neutron [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-eb224207-5b66-43ba-8824-f4198c246d6d req-7e5325e3-3188-4dbc-a02e-d1f5765c12f0 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Received event network-changed-7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.compute.manager [req-eb224207-5b66-43ba-8824-f4198c246d6d req-7e5325e3-3188-4dbc-a02e-d1f5765c12f0 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Refreshing instance network info cache due to event network-changed-7f00e811-6655-498b-a09b-5a45aba618b4. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-eb224207-5b66-43ba-8824-f4198c246d6d req-7e5325e3-3188-4dbc-a02e-d1f5765c12f0 service nova] Acquiring lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.network.neutron [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.network.neutron [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Updating instance_info_cache with network_info: [{"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Releasing lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.compute.manager [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Instance network_info: |[{"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-eb224207-5b66-43ba-8824-f4198c246d6d req-7e5325e3-3188-4dbc-a02e-d1f5765c12f0 service nova] Acquired lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.network.neutron [req-eb224207-5b66-43ba-8824-f4198c246d6d req-7e5325e3-3188-4dbc-a02e-d1f5765c12f0 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Refreshing network info cache for port 7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Start _get_guest_xml network_info=[{"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:45:04 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:45:04 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:45:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-441026141',display_name='tempest-AttachVolumeNegativeTest-server-441026141',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-441026141',id=46,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPq10jRsZwJAwjn8fWjAt5oHsMf2IPnRo2qmPAUD/9zrrvsoEgKc9ia04kVf6fotTh4Pv5yyx4MxgJSrLSbRN1rSGK9xMKoeETId4pSL+n09LaBGKTmlM0sgh6MwFAvp1A==',key_name='tempest-keypair-276811093',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='601e2806907a48d3995cc90b54da3c8c',ramdisk_id='',reservation_id='r-tcl7535t',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1197484356',owner_user_name='tempest-AttachVolumeNegativeTest-1197484356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:45:03Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7cab2461ab0d4eb08b62feaf6396bc87',uuid=f5a130f4-02c3-41fc-bbdb-d2b844e1320b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converting VIF {"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1a:e9:5f,bridge_name='br-int',has_traffic_filtering=True,id=7f00e811-6655-498b-a09b-5a45aba618b4,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f00e811-66') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.objects.instance [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'pci_devices' on Instance uuid f5a130f4-02c3-41fc-bbdb-d2b844e1320b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:45:04 user nova-compute[71317]:   <uuid>f5a130f4-02c3-41fc-bbdb-d2b844e1320b</uuid>
Apr 14 04:45:04 user nova-compute[71317]:   <name>instance-0000002e</name>
Apr 14 04:45:04 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:45:04 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:45:04 user nova-compute[71317]:   <metadata>
Apr 14 04:45:04 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:45:04 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:45:04 user nova-compute[71317]:       <nova:name>tempest-AttachVolumeNegativeTest-server-441026141</nova:name>
Apr 14 04:45:04 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:45:04</nova:creationTime>
Apr 14 04:45:04 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:45:04 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:45:04 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:45:04 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:45:04 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:45:04 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:45:04 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:45:04 user nova-compute[71317]:       <nova:owner>
Apr 14 04:45:04 user nova-compute[71317]:         <nova:user uuid="7cab2461ab0d4eb08b62feaf6396bc87">tempest-AttachVolumeNegativeTest-1197484356-project-member</nova:user>
Apr 14 04:45:04 user nova-compute[71317]:         <nova:project uuid="601e2806907a48d3995cc90b54da3c8c">tempest-AttachVolumeNegativeTest-1197484356</nova:project>
Apr 14 04:45:04 user nova-compute[71317]:       </nova:owner>
Apr 14 04:45:04 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:45:04 user nova-compute[71317]:       <nova:ports>
Apr 14 04:45:04 user nova-compute[71317]:         <nova:port uuid="7f00e811-6655-498b-a09b-5a45aba618b4">
Apr 14 04:45:04 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 14 04:45:04 user nova-compute[71317]:         </nova:port>
Apr 14 04:45:04 user nova-compute[71317]:       </nova:ports>
Apr 14 04:45:04 user nova-compute[71317]:     </nova:instance>
Apr 14 04:45:04 user nova-compute[71317]:   </metadata>
Apr 14 04:45:04 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:45:04 user nova-compute[71317]:     <system>
Apr 14 04:45:04 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:45:04 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:45:04 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:45:04 user nova-compute[71317]:       <entry name="serial">f5a130f4-02c3-41fc-bbdb-d2b844e1320b</entry>
Apr 14 04:45:04 user nova-compute[71317]:       <entry name="uuid">f5a130f4-02c3-41fc-bbdb-d2b844e1320b</entry>
Apr 14 04:45:04 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:45:04 user nova-compute[71317]:     </system>
Apr 14 04:45:04 user nova-compute[71317]:   </sysinfo>
Apr 14 04:45:04 user nova-compute[71317]:   <os>
Apr 14 04:45:04 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:45:04 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:45:04 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:45:04 user nova-compute[71317]:   </os>
Apr 14 04:45:04 user nova-compute[71317]:   <features>
Apr 14 04:45:04 user nova-compute[71317]:     <acpi/>
Apr 14 04:45:04 user nova-compute[71317]:     <apic/>
Apr 14 04:45:04 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:45:04 user nova-compute[71317]:   </features>
Apr 14 04:45:04 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:45:04 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:45:04 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:45:04 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:45:04 user nova-compute[71317]:   </clock>
Apr 14 04:45:04 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:45:04 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:45:04 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:45:04 user nova-compute[71317]:   </cpu>
Apr 14 04:45:04 user nova-compute[71317]:   <devices>
Apr 14 04:45:04 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:45:04 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:45:04 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk"/>
Apr 14 04:45:04 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:45:04 user nova-compute[71317]:     </disk>
Apr 14 04:45:04 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:45:04 user nova-compute[71317]:       <mac address="fa:16:3e:1a:e9:5f"/>
Apr 14 04:45:04 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:45:04 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:45:04 user nova-compute[71317]:       <target dev="tap7f00e811-66"/>
Apr 14 04:45:04 user nova-compute[71317]:     </interface>
Apr 14 04:45:04 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:45:04 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/console.log" append="off"/>
Apr 14 04:45:04 user nova-compute[71317]:     </serial>
Apr 14 04:45:04 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:45:04 user nova-compute[71317]:     <video>
Apr 14 04:45:04 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:45:04 user nova-compute[71317]:     </video>
Apr 14 04:45:04 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:45:04 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:45:04 user nova-compute[71317]:     </rng>
Apr 14 04:45:04 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:45:04 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:45:04 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:45:04 user nova-compute[71317]:     </memballoon>
Apr 14 04:45:04 user nova-compute[71317]:   </devices>
Apr 14 04:45:04 user nova-compute[71317]: </domain>
Apr 14 04:45:04 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:45:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-441026141',display_name='tempest-AttachVolumeNegativeTest-server-441026141',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-441026141',id=46,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPq10jRsZwJAwjn8fWjAt5oHsMf2IPnRo2qmPAUD/9zrrvsoEgKc9ia04kVf6fotTh4Pv5yyx4MxgJSrLSbRN1rSGK9xMKoeETId4pSL+n09LaBGKTmlM0sgh6MwFAvp1A==',key_name='tempest-keypair-276811093',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='601e2806907a48d3995cc90b54da3c8c',ramdisk_id='',reservation_id='r-tcl7535t',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1197484356',owner_user_name='tempest-AttachVolumeNegativeTest-1197484356-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:45:03Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7cab2461ab0d4eb08b62feaf6396bc87',uuid=f5a130f4-02c3-41fc-bbdb-d2b844e1320b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converting VIF {"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1a:e9:5f,bridge_name='br-int',has_traffic_filtering=True,id=7f00e811-6655-498b-a09b-5a45aba618b4,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f00e811-66') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG os_vif [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:1a:e9:5f,bridge_name='br-int',has_traffic_filtering=True,id=7f00e811-6655-498b-a09b-5a45aba618b4,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f00e811-66') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7f00e811-66, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap7f00e811-66, col_values=(('external_ids', {'iface-id': '7f00e811-6655-498b-a09b-5a45aba618b4', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:1a:e9:5f', 'vm-uuid': 'f5a130f4-02c3-41fc-bbdb-d2b844e1320b'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:04 user nova-compute[71317]: INFO os_vif [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:1a:e9:5f,bridge_name='br-int',has_traffic_filtering=True,id=7f00e811-6655-498b-a09b-5a45aba618b4,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f00e811-66')
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:45:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No VIF found with MAC fa:16:3e:1a:e9:5f, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:45:05 user nova-compute[71317]: DEBUG nova.network.neutron [req-eb224207-5b66-43ba-8824-f4198c246d6d req-7e5325e3-3188-4dbc-a02e-d1f5765c12f0 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Updated VIF entry in instance network info cache for port 7f00e811-6655-498b-a09b-5a45aba618b4. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:45:05 user nova-compute[71317]: DEBUG nova.network.neutron [req-eb224207-5b66-43ba-8824-f4198c246d6d req-7e5325e3-3188-4dbc-a02e-d1f5765c12f0 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Updating instance_info_cache with network_info: [{"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:45:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-eb224207-5b66-43ba-8824-f4198c246d6d req-7e5325e3-3188-4dbc-a02e-d1f5765c12f0 service nova] Releasing lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:45:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-a4bfcbf9-fa37-489c-9025-1af667e87daf req-0e53e57e-8d26-42f2-82de-906abe2a83f0 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Received event network-vif-plugged-7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:45:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a4bfcbf9-fa37-489c-9025-1af667e87daf req-0e53e57e-8d26-42f2-82de-906abe2a83f0 service nova] Acquiring lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a4bfcbf9-fa37-489c-9025-1af667e87daf req-0e53e57e-8d26-42f2-82de-906abe2a83f0 service nova] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:06 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a4bfcbf9-fa37-489c-9025-1af667e87daf req-0e53e57e-8d26-42f2-82de-906abe2a83f0 service nova] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:06 user nova-compute[71317]: DEBUG nova.compute.manager [req-a4bfcbf9-fa37-489c-9025-1af667e87daf req-0e53e57e-8d26-42f2-82de-906abe2a83f0 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] No waiting events found dispatching network-vif-plugged-7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:45:06 user nova-compute[71317]: WARNING nova.compute.manager [req-a4bfcbf9-fa37-489c-9025-1af667e87daf req-0e53e57e-8d26-42f2-82de-906abe2a83f0 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Received unexpected event network-vif-plugged-7f00e811-6655-498b-a09b-5a45aba618b4 for instance with vm_state building and task_state spawning.
Apr 14 04:45:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447508.3606694, f5a130f4-02c3-41fc-bbdb-d2b844e1320b => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:45:08 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] VM Resumed (Lifecycle Event)
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:45:08 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Instance spawned successfully.
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:45:08 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447508.3630404, f5a130f4-02c3-41fc-bbdb-d2b844e1320b => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:45:08 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] VM Started (Lifecycle Event)
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:45:08 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:45:08 user nova-compute[71317]: INFO nova.compute.manager [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Took 5.46 seconds to spawn the instance on the hypervisor.
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-c44f9cfe-11b6-465f-a824-4b488df6d2ac req-1c458cab-39fe-4994-83b5-28c740eb2d54 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Received event network-vif-plugged-7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c44f9cfe-11b6-465f-a824-4b488df6d2ac req-1c458cab-39fe-4994-83b5-28c740eb2d54 service nova] Acquiring lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c44f9cfe-11b6-465f-a824-4b488df6d2ac req-1c458cab-39fe-4994-83b5-28c740eb2d54 service nova] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c44f9cfe-11b6-465f-a824-4b488df6d2ac req-1c458cab-39fe-4994-83b5-28c740eb2d54 service nova] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG nova.compute.manager [req-c44f9cfe-11b6-465f-a824-4b488df6d2ac req-1c458cab-39fe-4994-83b5-28c740eb2d54 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] No waiting events found dispatching network-vif-plugged-7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:45:08 user nova-compute[71317]: WARNING nova.compute.manager [req-c44f9cfe-11b6-465f-a824-4b488df6d2ac req-1c458cab-39fe-4994-83b5-28c740eb2d54 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Received unexpected event network-vif-plugged-7f00e811-6655-498b-a09b-5a45aba618b4 for instance with vm_state building and task_state spawning.
Apr 14 04:45:08 user nova-compute[71317]: INFO nova.compute.manager [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Took 6.03 seconds to build instance.
Apr 14 04:45:08 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-57ce5ea8-3bb0-4b17-9243-37af467a1b49 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.170s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:45:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:45:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:36 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:42 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:45:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:45:46 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:45:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:45:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:45:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:45:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:45:46 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:45:46 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lazy-loading 'info_cache' on Instance uuid f5a130f4-02c3-41fc-bbdb-d2b844e1320b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:45:46 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Updating instance_info_cache with network_info: [{"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:45:46 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:45:46 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:45:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "77f6bb83-54ab-485e-8943-742f2f576607" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:47 user nova-compute[71317]: INFO nova.compute.manager [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Unshelving
Apr 14 04:45:47 user nova-compute[71317]: INFO nova.virt.block_device [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Booting with volume a8e6a311-8ecf-435d-a4bd-23726576f9bb at /dev/vdc
Apr 14 04:45:47 user nova-compute[71317]: DEBUG os_brick.utils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:45:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:47 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[19e9cf1e-ac48-48f8-be96-e9d343c6f21b]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:47 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcff1b7a-33d7-48a0-ab69-1ca853677fc8]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:47 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ec07e06-4891-4c72-8ddf-e685590a966c]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0366057-0a53-46fc-bdd5-1abf837cca50]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:45:48 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:45:48 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3b7b86a-40cd-41ba-85a4-304a91ccdc2a]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG os_brick.utils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] <== get_connector_properties: return (128ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updating existing volume attachment record: d1917683-8961-4777-a164-2cfbf7e4477f {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:45:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:45:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:45:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:52 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:45:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:52 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'pci_requests' on Instance uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:45:52 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'numa_topology' on Instance uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:45:52 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:45:52 user nova-compute[71317]: INFO nova.compute.claims [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Claim successful on node user
Apr 14 04:45:52 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:45:52 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:45:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.250s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:52 user nova-compute[71317]: INFO nova.network.neutron [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updating port 440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Apr 14 04:45:53 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:45:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:45:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquired lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:45:53 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:45:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-509320e5-d045-437e-bba0-790cab5eb534 req-623c5845-8e00-4530-872d-d038a0019acb service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-changed-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:45:53 user nova-compute[71317]: DEBUG nova.compute.manager [req-509320e5-d045-437e-bba0-790cab5eb534 req-623c5845-8e00-4530-872d-d038a0019acb service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Refreshing instance network info cache due to event network-changed-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:45:53 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-509320e5-d045-437e-bba0-790cab5eb534 req-623c5845-8e00-4530-872d-d038a0019acb service nova] Acquiring lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG nova.network.neutron [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updating instance_info_cache with network_info: [{"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Releasing lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:45:54 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Creating image(s)
Apr 14 04:45:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "/opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "/opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "/opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'trusted_certs' on Instance uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-509320e5-d045-437e-bba0-790cab5eb534 req-623c5845-8e00-4530-872d-d038a0019acb service nova] Acquired lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-509320e5-d045-437e-bba0-790cab5eb534 req-623c5845-8e00-4530-872d-d038a0019acb service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Refreshing network info cache for port 440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "8ac6af1c271197b8c863a86553be324e1abbd658" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "8ac6af1c271197b8c863a86553be324e1abbd658" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658.part --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json" returned: 0 in 0.906s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658.part --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG nova.virt.images [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] ed0f5b6b-1ab3-461f-987b-1c736a691553 was qcow2, converting to raw {{(pid=71317) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658.part /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658.converted {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658.part /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658.converted" returned: 0 in 0.576s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658.converted --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:55 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:45:55 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:45:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8666MB free_disk=26.283828735351562GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance f5a130f4-02c3-41fc-bbdb-d2b844e1320b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Instance with task_state "spawning" is not being actively managed by this compute host but has allocations referencing this compute node (e35a99e4-b1ea-4792-a9ff-f93045b2018b): {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. Skipping heal of allocations during the task state transition. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1705}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658.converted --force-share --output=json" returned: 0 in 0.146s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "8ac6af1c271197b8c863a86553be324e1abbd658" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.830s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658 --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "8ac6af1c271197b8c863a86553be324e1abbd658" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "8ac6af1c271197b8c863a86553be324e1abbd658" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-509320e5-d045-437e-bba0-790cab5eb534 req-623c5845-8e00-4530-872d-d038a0019acb service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updated VIF entry in instance network info cache for port 440e7ef0-4f4c-4923-ab57-8e1ba56dbea3. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG nova.network.neutron [req-509320e5-d045-437e-bba0-790cab5eb534 req-623c5845-8e00-4530-872d-d038a0019acb service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updating instance_info_cache with network_info: [{"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.302s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-509320e5-d045-437e-bba0-790cab5eb534 req-623c5845-8e00-4530-872d-d038a0019acb service nova] Releasing lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658 --force-share --output=json" returned: 0 in 0.137s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658,backing_fmt=raw /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658,backing_fmt=raw /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk 1073741824" returned: 0 in 0.049s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "8ac6af1c271197b8c863a86553be324e1abbd658" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.192s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658 --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'migration_context' on Instance uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:45:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Rebasing disk image.
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): qemu-img rebase -b /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d -F raw /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:57 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:45:57 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:45:57 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "qemu-img rebase -b /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d -F raw /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk" returned: 0 in 1.959s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Ensure instance console log exists: /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Start _get_guest_xml network_info=[{"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} image_meta=ImageMeta(checksum='d1a5d5f2f8aceb7d239079f9c231fc6f',container_format='bare',created_at=2023-04-14T04:43:55Z,direct_url=<?>,disk_format='qcow2',id=ed0f5b6b-1ab3-461f-987b-1c736a691553,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-1249800375-shelved',owner='c10c8cf348e04e44afbaae93a4b60e36',properties=ImageMetaProps,protected=<?>,size=38600704,status='active',tags=<?>,updated_at=2023-04-14T04:43:59Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [{'disk_bus': 'virtio', 'boot_index': None, 'delete_on_termination': False, 'guest_format': None, 'device_type': 'disk', 'mount_device': '/dev/vdc', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ifwOWKGs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attached', 'instance': '77f6bb83-54ab-485e-8943-742f2f576607', 'attached_at': '', 'detached_at': '', 'volume_id': 'a8e6a311-8ecf-435d-a4bd-23726576f9bb', 'serial': 'a8e6a311-8ecf-435d-a4bd-23726576f9bb'}, 'attachment_id': 'd1917683-8961-4777-a164-2cfbf7e4477f', 'volume_type': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:45:58 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:45:58 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='d1a5d5f2f8aceb7d239079f9c231fc6f',container_format='bare',created_at=2023-04-14T04:43:55Z,direct_url=<?>,disk_format='qcow2',id=ed0f5b6b-1ab3-461f-987b-1c736a691553,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-1249800375-shelved',owner='c10c8cf348e04e44afbaae93a4b60e36',properties=ImageMetaProps,protected=<?>,size=38600704,status='active',tags=<?>,updated_at=2023-04-14T04:43:59Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'vcpu_model' on Instance uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ifwOWKGs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:45:58 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:45:58 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:45:58 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9537242e-52b8-4576-8988-287d36e26210]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:45:58 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[689ba08d-a93f-493a-9da3-6d8321edf53d]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb] added
Apr 14 04:45:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a5115bc-f33e-4565-8b3b-41c3bb601281]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.027s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[05db0e1e-795f-406d-bafb-78ed6bb44429]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ifwOWKGs {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ifwOWKGs" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5dd0299a-c013-450b-8c5f-e5c81c3539b7]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'ifwOWKGs'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f732f3da-5246-437e-a099-578037601d42]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[c38ab866-cc06-4d6e-b1b2-7e1129645b48]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 14 04:45:58 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 14 04:45:58 user nova-compute[71317]:  {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:45:58 user nova-compute[71317]: WARNING os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 14 04:45:58 user nova-compute[71317]: 
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --login" returned: 0 in 0.045s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[364c2d80-68b1-447f-9db2-f94894544d15]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb, portal: 172.16.0.220,3260]
Apr 14 04:45:58 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb, portal: 172.16.0.220,3260] successful.
Apr 14 04:45:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[6880dde8-bfae-445f-bffe-520303ed9515]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[5889b136-37ba-4ee2-809d-cd98f5206101]: (4, ('tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)
Apr 14 04:45:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsi session list stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)
Apr 14 04:45:58 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.257s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] HCTL ('33', '-', '-', 0) found on session 53 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2963226-5ea0-4488-b6e8-7c0359ad7230]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:45:58 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Searching for a device in session 53 and hctl ['33', '*', '*', 0] yield: None {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:45:59 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Searching for a device in session 53 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:45:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ifwOWKGs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:45:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.286s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:45:59 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] <== connect_volume: return (1286ms) {'type': 'block', 'scsi_wwn': '26636383637643166', 'path': '/dev/sdb'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:45:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26636383637643166', 'path': '/dev/sdb'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:42:00Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1249800375',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1249800375',id=44,image_ref='ed0f5b6b-1ab3-461f-987b-1c736a691553',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-keypair-1996810505',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:42:07Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='c10c8cf348e04e44afbaae93a4b60e36',ramdisk_id='',reservation_id='r-4s0bcxm4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2141578557',owner_user_name='tempest-AttachVolumeShelveTestJSON-2141578557-project-member',shelved_at='2023-04-14T04:43:58.799231',shelved_host='user',shelved_image_id='ed0f5b6b-1ab3-461f-987b-1c736a691553'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:45:48Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='67d6c699fc524554a6f4983af91d58fc',uuid=77f6bb83-54ab-485e-8943-742f2f576607,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converting VIF {"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'pci_devices' on Instance uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:46:00 user nova-compute[71317]:   <uuid>77f6bb83-54ab-485e-8943-742f2f576607</uuid>
Apr 14 04:46:00 user nova-compute[71317]:   <name>instance-0000002c</name>
Apr 14 04:46:00 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:46:00 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:46:00 user nova-compute[71317]:   <metadata>
Apr 14 04:46:00 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:46:00 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:46:00 user nova-compute[71317]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-1249800375</nova:name>
Apr 14 04:46:00 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:45:58</nova:creationTime>
Apr 14 04:46:00 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:46:00 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:46:00 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:46:00 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:46:00 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:46:00 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:46:00 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:46:00 user nova-compute[71317]:       <nova:owner>
Apr 14 04:46:00 user nova-compute[71317]:         <nova:user uuid="67d6c699fc524554a6f4983af91d58fc">tempest-AttachVolumeShelveTestJSON-2141578557-project-member</nova:user>
Apr 14 04:46:00 user nova-compute[71317]:         <nova:project uuid="c10c8cf348e04e44afbaae93a4b60e36">tempest-AttachVolumeShelveTestJSON-2141578557</nova:project>
Apr 14 04:46:00 user nova-compute[71317]:       </nova:owner>
Apr 14 04:46:00 user nova-compute[71317]:       <nova:root type="image" uuid="ed0f5b6b-1ab3-461f-987b-1c736a691553"/>
Apr 14 04:46:00 user nova-compute[71317]:       <nova:ports>
Apr 14 04:46:00 user nova-compute[71317]:         <nova:port uuid="440e7ef0-4f4c-4923-ab57-8e1ba56dbea3">
Apr 14 04:46:00 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Apr 14 04:46:00 user nova-compute[71317]:         </nova:port>
Apr 14 04:46:00 user nova-compute[71317]:       </nova:ports>
Apr 14 04:46:00 user nova-compute[71317]:     </nova:instance>
Apr 14 04:46:00 user nova-compute[71317]:   </metadata>
Apr 14 04:46:00 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:46:00 user nova-compute[71317]:     <system>
Apr 14 04:46:00 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:46:00 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:46:00 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:46:00 user nova-compute[71317]:       <entry name="serial">77f6bb83-54ab-485e-8943-742f2f576607</entry>
Apr 14 04:46:00 user nova-compute[71317]:       <entry name="uuid">77f6bb83-54ab-485e-8943-742f2f576607</entry>
Apr 14 04:46:00 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:46:00 user nova-compute[71317]:     </system>
Apr 14 04:46:00 user nova-compute[71317]:   </sysinfo>
Apr 14 04:46:00 user nova-compute[71317]:   <os>
Apr 14 04:46:00 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:46:00 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:46:00 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:46:00 user nova-compute[71317]:   </os>
Apr 14 04:46:00 user nova-compute[71317]:   <features>
Apr 14 04:46:00 user nova-compute[71317]:     <acpi/>
Apr 14 04:46:00 user nova-compute[71317]:     <apic/>
Apr 14 04:46:00 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:46:00 user nova-compute[71317]:   </features>
Apr 14 04:46:00 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:46:00 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:46:00 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:46:00 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:46:00 user nova-compute[71317]:   </clock>
Apr 14 04:46:00 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:46:00 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:46:00 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:46:00 user nova-compute[71317]:   </cpu>
Apr 14 04:46:00 user nova-compute[71317]:   <devices>
Apr 14 04:46:00 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:46:00 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:46:00 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk"/>
Apr 14 04:46:00 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:46:00 user nova-compute[71317]:     </disk>
Apr 14 04:46:00 user nova-compute[71317]:     <disk type="block" device="disk">
Apr 14 04:46:00 user nova-compute[71317]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:46:00 user nova-compute[71317]:       <source dev="/dev/sdb"/>
Apr 14 04:46:00 user nova-compute[71317]:       <target dev="vdc" bus="virtio"/>
Apr 14 04:46:00 user nova-compute[71317]:       <serial>a8e6a311-8ecf-435d-a4bd-23726576f9bb</serial>
Apr 14 04:46:00 user nova-compute[71317]:     </disk>
Apr 14 04:46:00 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:46:00 user nova-compute[71317]:       <mac address="fa:16:3e:6b:b2:cc"/>
Apr 14 04:46:00 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:46:00 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:46:00 user nova-compute[71317]:       <target dev="tap440e7ef0-4f"/>
Apr 14 04:46:00 user nova-compute[71317]:     </interface>
Apr 14 04:46:00 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:46:00 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/console.log" append="off"/>
Apr 14 04:46:00 user nova-compute[71317]:     </serial>
Apr 14 04:46:00 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:46:00 user nova-compute[71317]:     <video>
Apr 14 04:46:00 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:46:00 user nova-compute[71317]:     </video>
Apr 14 04:46:00 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:46:00 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:46:00 user nova-compute[71317]:     </rng>
Apr 14 04:46:00 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:46:00 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:46:00 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:46:00 user nova-compute[71317]:     </memballoon>
Apr 14 04:46:00 user nova-compute[71317]:   </devices>
Apr 14 04:46:00 user nova-compute[71317]: </domain>
Apr 14 04:46:00 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:42:00Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1249800375',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1249800375',id=44,image_ref='ed0f5b6b-1ab3-461f-987b-1c736a691553',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-keypair-1996810505',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:42:07Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='c10c8cf348e04e44afbaae93a4b60e36',ramdisk_id='',reservation_id='r-4s0bcxm4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2141578557',owner_user_name='tempest-AttachVolumeShelveTestJSON-2141578557-project-member',shelved_at='2023-04-14T04:43:58.799231',shelved_host='user',shelved_image_id='ed0f5b6b-1ab3-461f-987b-1c736a691553'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:45:48Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='67d6c699fc524554a6f4983af91d58fc',uuid=77f6bb83-54ab-485e-8943-742f2f576607,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converting VIF {"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG os_vif [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap440e7ef0-4f, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap440e7ef0-4f, col_values=(('external_ids', {'iface-id': '440e7ef0-4f4c-4923-ab57-8e1ba56dbea3', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6b:b2:cc', 'vm-uuid': '77f6bb83-54ab-485e-8943-742f2f576607'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:00 user nova-compute[71317]: INFO os_vif [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f')
Apr 14 04:46:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] No BDM found with device name vdc, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:46:00 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] No VIF found with MAC fa:16:3e:6b:b2:cc, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:46:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-0ccd3e0d-678a-4488-a26a-6b8a150b26e2 req-d779eb42-43dc-48bf-b667-e64a46d76ccf service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:46:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0ccd3e0d-678a-4488-a26a-6b8a150b26e2 req-d779eb42-43dc-48bf-b667-e64a46d76ccf service nova] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:46:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0ccd3e0d-678a-4488-a26a-6b8a150b26e2 req-d779eb42-43dc-48bf-b667-e64a46d76ccf service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:46:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-0ccd3e0d-678a-4488-a26a-6b8a150b26e2 req-d779eb42-43dc-48bf-b667-e64a46d76ccf service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:46:01 user nova-compute[71317]: DEBUG nova.compute.manager [req-0ccd3e0d-678a-4488-a26a-6b8a150b26e2 req-d779eb42-43dc-48bf-b667-e64a46d76ccf service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] No waiting events found dispatching network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:46:01 user nova-compute[71317]: WARNING nova.compute.manager [req-0ccd3e0d-678a-4488-a26a-6b8a150b26e2 req-d779eb42-43dc-48bf-b667-e64a46d76ccf service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received unexpected event network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 for instance with vm_state shelved_offloaded and task_state spawning.
Apr 14 04:46:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:03 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447563.6841104, 77f6bb83-54ab-485e-8943-742f2f576607 => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:46:03 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] VM Resumed (Lifecycle Event)
Apr 14 04:46:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:46:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:46:03 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Instance spawned successfully.
Apr 14 04:46:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:46:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:46:03 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:46:03 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447563.6865652, 77f6bb83-54ab-485e-8943-742f2f576607 => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:46:03 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] VM Started (Lifecycle Event)
Apr 14 04:46:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:46:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Synchronizing instance power state after lifecycle event "Started"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:46:03 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:46:03 user nova-compute[71317]: DEBUG nova.compute.manager [req-400036bd-d3a6-4522-b373-f5df4cae0130 req-5f33be16-d4bb-4f54-a0e1-8970a62fd068 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:46:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-400036bd-d3a6-4522-b373-f5df4cae0130 req-5f33be16-d4bb-4f54-a0e1-8970a62fd068 service nova] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:46:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-400036bd-d3a6-4522-b373-f5df4cae0130 req-5f33be16-d4bb-4f54-a0e1-8970a62fd068 service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:46:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-400036bd-d3a6-4522-b373-f5df4cae0130 req-5f33be16-d4bb-4f54-a0e1-8970a62fd068 service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:46:03 user nova-compute[71317]: DEBUG nova.compute.manager [req-400036bd-d3a6-4522-b373-f5df4cae0130 req-5f33be16-d4bb-4f54-a0e1-8970a62fd068 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] No waiting events found dispatching network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:46:03 user nova-compute[71317]: WARNING nova.compute.manager [req-400036bd-d3a6-4522-b373-f5df4cae0130 req-5f33be16-d4bb-4f54-a0e1-8970a62fd068 service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received unexpected event network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 for instance with vm_state shelved_offloaded and task_state spawning.
Apr 14 04:46:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:46:03 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2df75a5e-ccce-41fe-9383-261dbdc41c00 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "77f6bb83-54ab-485e-8943-742f2f576607" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 16.196s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:46:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:46:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:46:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:46:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:46:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:46:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:46:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:44 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:46:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:46:47 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:46:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:46:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:46:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:46:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:46:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:46:47 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lazy-loading 'info_cache' on Instance uuid f5a130f4-02c3-41fc-bbdb-d2b844e1320b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:46:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Updating instance_info_cache with network_info: [{"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:46:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:46:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:46:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:46:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:50 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:46:52 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:46:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-78c3d0e7-59ee-4e89-8928-a4ab215cbd72 req-22cd36a3-a949-45b2-ae0f-15227891171f service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Received event network-changed-7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG nova.compute.manager [req-78c3d0e7-59ee-4e89-8928-a4ab215cbd72 req-22cd36a3-a949-45b2-ae0f-15227891171f service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Refreshing instance network info cache due to event network-changed-7f00e811-6655-498b-a09b-5a45aba618b4. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-78c3d0e7-59ee-4e89-8928-a4ab215cbd72 req-22cd36a3-a949-45b2-ae0f-15227891171f service nova] Acquiring lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-78c3d0e7-59ee-4e89-8928-a4ab215cbd72 req-22cd36a3-a949-45b2-ae0f-15227891171f service nova] Acquired lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-78c3d0e7-59ee-4e89-8928-a4ab215cbd72 req-22cd36a3-a949-45b2-ae0f-15227891171f service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Refreshing network info cache for port 7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-78c3d0e7-59ee-4e89-8928-a4ab215cbd72 req-22cd36a3-a949-45b2-ae0f-15227891171f service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Updated VIF entry in instance network info cache for port 7f00e811-6655-498b-a09b-5a45aba618b4. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG nova.network.neutron [req-78c3d0e7-59ee-4e89-8928-a4ab215cbd72 req-22cd36a3-a949-45b2-ae0f-15227891171f service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Updating instance_info_cache with network_info: [{"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.114", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-78c3d0e7-59ee-4e89-8928-a4ab215cbd72 req-22cd36a3-a949-45b2-ae0f-15227891171f service nova] Releasing lock "refresh_cache-f5a130f4-02c3-41fc-bbdb-d2b844e1320b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:46:54 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] skipping disk /dev/sdb (vdc) as it is a volume {{(pid=71317) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 04:46:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:46:55 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:46:55 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:46:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8659MB free_disk=26.257511138916016GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:46:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:46:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:46:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance f5a130f4-02c3-41fc-bbdb-d2b844e1320b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:46:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 77f6bb83-54ab-485e-8943-742f2f576607 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:46:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:46:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=768MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:46:55 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:46:55 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:46:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:46:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.232s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG nova.objects.instance [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'flavor' on Instance uuid f5a130f4-02c3-41fc-bbdb-d2b844e1320b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.054s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:46:56 user nova-compute[71317]: INFO nova.compute.manager [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Attaching volume 39ce02ae-5c89-4be1-abd2-8c275a9f7af2 to /dev/vdb
Apr 14 04:46:56 user nova-compute[71317]: DEBUG os_brick.utils [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbfedb1d-e74a-4dc5-bb37-b35c0e72f93f]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:66f2ad7cf0e5\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e433bebd-0507-46bc-bf0c-df8e4cc3b145]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9220ef74-c9db-4eb8-bb33-74a40a1566d5]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f15b5c5-7c80-4456-bc95-f96eb87af211]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Running cmd (subprocess): nvme version {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] 'nvme version' failed. Not Retrying. {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] nvme not present on system {{(pid=71317) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 04:46:56 user nova-compute[71317]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 04:46:56 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a83f33fa-fe80-41be-b3c6-10b6e16afff1]: (4, '') {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71317) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.lightos [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] LIGHTOS: no hostnqn found. {{(pid=71317) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG os_brick.utils [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] <== get_connector_properties: return (126ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:66f2ad7cf0e5', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:46:56 user nova-compute[71317]: DEBUG nova.virt.block_device [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Updating existing volume attachment record: 3a49c816-650e-454d-9aa5-e6dee356cb7c {{(pid=71317) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 04:46:57 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:46:57 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:46:57 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:46:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:00 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:47:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Calling os-brick to attach iSCSI Volume {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CwucHxdV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:47:03 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:47:03 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:47:03 user nova-compute[71317]: INFO os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ff2771fe-8779-4f8a-a5ce-233745d220fa]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 04:47:03 user nova-compute[71317]:  {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --interface default --op new {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c41f4a0-1440-4ef3-b1ca-0b2a202284e0]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2] added\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2] added
Apr 14 04:47:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f8c2d3c-feb4-45f2-84c0-6cb6b8f42ea4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[cef94ca3-6b43-41cc-b3c9-c07070ae6ebc]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v CwucHxdV {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v CwucHxdV" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a211adcd-2a4d-43f9-89b3-e252c442ac6c]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'CwucHxdV'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b40be0cb-7d89-4355-b289-b4fc4199a975]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a6b69ed-f39a-473e-87b8-5e9bb4973dda]: (4, ('tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)
Apr 14 04:47:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsi session list stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)
Apr 14 04:47:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --login {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --login" returned: 0 in 0.049s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8877e857-93af-4514-9fd6-e655d0cc6e4a]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2, portal: 172.16.0.220,3260]
Apr 14 04:47:03 user nova-compute[71317]: Login to [iface: default, target: iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2, portal: 172.16.0.220,3260] successful.
Apr 14 04:47:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[d80fd1c3-3fd0-418a-9337-fa0a804cd182]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[865eeb84-6398-4b82-adb3-9464632c4caa]: (4, ('tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)
Apr 14 04:47:03 user nova-compute[71317]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 (non-flash)
Apr 14 04:47:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsi session list stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)
Apr 14 04:47:03 user nova-compute[71317]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 (non-flash)
Apr 14 04:47:03 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.254s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Connected to 172.16.0.220:3260 {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] HCTL ('34', '-', '-', 0) found on session 54 with lun 0 {{(pid=71317) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71317) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ad59039-43f4-4107-9a74-4ba4fd43cac6]: (4, ('- - 0', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Searching for a device in session 54 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71317) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CwucHxdV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71317) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 04:47:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.284s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:47:04 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] <== connect_volume: return (1285ms) {'type': 'block', 'scsi_wwn': '26364383532363939', 'path': '/dev/sdc'} {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:47:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26364383532363939', 'path': '/dev/sdc'} {{(pid=71317) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 04:47:04 user nova-compute[71317]: DEBUG nova.objects.instance [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'flavor' on Instance uuid f5a130f4-02c3-41fc-bbdb-d2b844e1320b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:47:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 04:47:04 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:47:04 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:47:04 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:47:04 user nova-compute[71317]:   <serial>39ce02ae-5c89-4be1-abd2-8c275a9f7af2</serial>
Apr 14 04:47:04 user nova-compute[71317]: </disk>
Apr 14 04:47:04 user nova-compute[71317]:  {{(pid=71317) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 04:47:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:47:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:47:04 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] No VIF found with MAC fa:16:3e:1a:e9:5f, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:47:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-4d9c49c9-e886-422e-80b9-3047d979622f tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 8.355s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:47:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:47:05 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:47:05 user nova-compute[71317]: INFO nova.compute.manager [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Detaching volume 39ce02ae-5c89-4be1-abd2-8c275a9f7af2
Apr 14 04:47:05 user nova-compute[71317]: INFO nova.virt.block_device [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Attempting to driver detach volume 39ce02ae-5c89-4be1-abd2-8c275a9f7af2 from mountpoint /dev/vdb
Apr 14 04:47:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Attempting to detach device vdb from instance f5a130f4-02c3-41fc-bbdb-d2b844e1320b from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:47:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:47:05 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:47:05 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:47:05 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:47:05 user nova-compute[71317]:   <serial>39ce02ae-5c89-4be1-abd2-8c275a9f7af2</serial>
Apr 14 04:47:05 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:47:05 user nova-compute[71317]: </disk>
Apr 14 04:47:05 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:47:05 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully detached device vdb from instance f5a130f4-02c3-41fc-bbdb-d2b844e1320b from the persistent domain config.
Apr 14 04:47:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f5a130f4-02c3-41fc-bbdb-d2b844e1320b from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:47:05 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:47:05 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:47:05 user nova-compute[71317]:   <source dev="/dev/sdc"/>
Apr 14 04:47:05 user nova-compute[71317]:   <target dev="vdb" bus="virtio"/>
Apr 14 04:47:05 user nova-compute[71317]:   <serial>39ce02ae-5c89-4be1-abd2-8c275a9f7af2</serial>
Apr 14 04:47:05 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 04:47:05 user nova-compute[71317]: </disk>
Apr 14 04:47:05 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681447626.152199, f5a130f4-02c3-41fc-bbdb-d2b844e1320b => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f5a130f4-02c3-41fc-bbdb-d2b844e1320b {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:47:06 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully detached device vdb from instance f5a130f4-02c3-41fc-bbdb-d2b844e1320b from the live domain config.
Apr 14 04:47:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CwucHxdV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:47:06 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[24de05a5-60b2-4c4b-a394-3b04adaec926]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:47:06 user nova-compute[71317]: No targets found.
Apr 14 04:47:06 user nova-compute[71317]: iSNS:
Apr 14 04:47:06 user nova-compute[71317]: No targets found.
Apr 14 04:47:06 user nova-compute[71317]: STATIC:
Apr 14 04:47:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb
Apr 14 04:47:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:47:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:47:06 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2
Apr 14 04:47:06 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:47:06 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:47:06 user nova-compute[71317]: FIRMWARE:
Apr 14 04:47:06 user nova-compute[71317]: No targets found.
Apr 14 04:47:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:47:06 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:47:06 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[656015c5-e8b7-4e72-85de-0f13ab748823]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[a56a02cf-9d70-4ebc-bf05-d8ef031ed671]: (4, ('tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)
Apr 14 04:47:06 user nova-compute[71317]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 (non-flash)
Apr 14 04:47:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsi session list stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)
Apr 14 04:47:06 user nova-compute[71317]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 (non-flash)
Apr 14 04:47:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f849076b130>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2'): ({'sdc'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Removing single pathed devices sdc {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[332129cc-7ba0-491f-afd0-230289a29e1e]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.025s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a97892e-1c22-4fe8-b01c-f37b17224811]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Flushing IO for device /dev/sdc {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.014s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[8372b7f5-4f74-4702-af39-4a3ead4687e9]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.038s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[763f8552-f8a0-4291-81b8-eddf30fa053e]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] SCSI volumes sdc have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8fb69e2-856f-459b-9351-50e095659bd6]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --logout" returned: 0 in 0.060s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[dfcd5d02-c3b8-4c8c-bc2c-3c19bb142310]: (4, ('Logging out of session [sid: 54, target: iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2, portal: 172.16.0.220,3260]\nLogout of [sid: 54, target: iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 54, target: iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2, portal: 172.16.0.220,3260]
Apr 14 04:47:06 user nova-compute[71317]: Logout of [sid: 54, target: iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2, portal: 172.16.0.220,3260] successful.
Apr 14 04:47:06 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:39ce02ae-5c89-4be1-abd2-8c275a9f7af2 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f7fb85b-d3e1-4513-bb06-0f411995ade4]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.301s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] <== disconnect_volume: return (303ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:47:06 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:47:07 user nova-compute[71317]: DEBUG nova.objects.instance [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'flavor' on Instance uuid f5a130f4-02c3-41fc-bbdb-d2b844e1320b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:47:07 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-6f78e05e-9c46-4c10-9c46-b496239a1516 tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.806s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:47:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:47:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:47:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:47:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:47:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:47:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:47:37 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "77f6bb83-54ab-485e-8943-742f2f576607" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:47:37 user nova-compute[71317]: INFO nova.compute.manager [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Detaching volume a8e6a311-8ecf-435d-a4bd-23726576f9bb
Apr 14 04:47:38 user nova-compute[71317]: INFO nova.virt.block_device [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Attempting to driver detach volume a8e6a311-8ecf-435d-a4bd-23726576f9bb from mountpoint /dev/vdc
Apr 14 04:47:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Attempting to detach device vdc from instance 77f6bb83-54ab-485e-8943-742f2f576607 from the persistent domain config. {{(pid=71317) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:47:38 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:47:38 user nova-compute[71317]:   <source dev="/dev/sdb"/>
Apr 14 04:47:38 user nova-compute[71317]:   <target dev="vdc" bus="virtio"/>
Apr 14 04:47:38 user nova-compute[71317]:   <serial>a8e6a311-8ecf-435d-a4bd-23726576f9bb</serial>
Apr 14 04:47:38 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 04:47:38 user nova-compute[71317]: </disk>
Apr 14 04:47:38 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:47:38 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Successfully detached device vdc from instance 77f6bb83-54ab-485e-8943-742f2f576607 from the persistent domain config.
Apr 14 04:47:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] (1/8): Attempting to detach device vdc with device alias virtio-disk2 from instance 77f6bb83-54ab-485e-8943-742f2f576607 from the live domain config. {{(pid=71317) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.guest [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 04:47:38 user nova-compute[71317]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 04:47:38 user nova-compute[71317]:   <source dev="/dev/sdb"/>
Apr 14 04:47:38 user nova-compute[71317]:   <target dev="vdc" bus="virtio"/>
Apr 14 04:47:38 user nova-compute[71317]:   <serial>a8e6a311-8ecf-435d-a4bd-23726576f9bb</serial>
Apr 14 04:47:38 user nova-compute[71317]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 04:47:38 user nova-compute[71317]: </disk>
Apr 14 04:47:38 user nova-compute[71317]:  {{(pid=71317) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Received event <DeviceRemovedEvent: 1681447658.4313655, 77f6bb83-54ab-485e-8943-742f2f576607 => virtio-disk2> from libvirt while the driver is waiting for it; dispatched. {{(pid=71317) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Start waiting for the detach event from libvirt for device vdc with device alias virtio-disk2 for instance 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 04:47:38 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Successfully detached device vdc from instance 77f6bb83-54ab-485e-8943-742f2f576607 from the live domain config.
Apr 14 04:47:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] calling os-brick to detach iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f84907c4160>, {'target_iqn': 'iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ifwOWKGs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 04:47:38 user nova-compute[71317]: WARNING os_brick.initiator.connectors.base [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[9480c090-3d0f-4875-b6a8-737484b7f3c2]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 04:47:38 user nova-compute[71317]: No targets found.
Apr 14 04:47:38 user nova-compute[71317]: iSNS:
Apr 14 04:47:38 user nova-compute[71317]: No targets found.
Apr 14 04:47:38 user nova-compute[71317]: STATIC:
Apr 14 04:47:38 user nova-compute[71317]: Target: iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb
Apr 14 04:47:38 user nova-compute[71317]:         Portal: 172.16.0.220:3260,-1
Apr 14 04:47:38 user nova-compute[71317]:                 Iface Name: default
Apr 14 04:47:38 user nova-compute[71317]: FIRMWARE:
Apr 14 04:47:38 user nova-compute[71317]: No targets found.
Apr 14 04:47:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 04:47:38 user nova-compute[71317]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 04:47:38 user nova-compute[71317]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71317) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb', 0)] {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.016s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe73fa32-9e16-449a-8761-95fa09ab1842]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[2fb0c1c8-d53f-4c36-b39e-3ff6bee43a8e]: (4, ('tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)
Apr 14 04:47:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsi session list stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb (non-flash)
Apr 14 04:47:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f849068d870>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb'): ({'sdb'}, set())}) {{(pid=71317) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Removing single pathed devices sdb {{(pid=71317) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[f000321d-1fe2-4867-acb7-6bd6dda94af2]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[142b543b-f49e-4a4e-a1eb-7b8e1166cec0]: (4, ('ok\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Flushing IO for device /dev/sdb {{(pid=71317) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[aaa8f433-8c54-4085-9f08-07b0307f3865]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71317) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.041s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[b19a8acc-1fb5-4a74-ace7-949a9c8b2937]: (4, ('1', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.linuxscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] SCSI volumes sdb have been removed. {{(pid=71317) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb')] {{(pid=71317) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a53c948-04cd-4cc9-ad14-7c3e40eeacca]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --logout {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --logout" returned: 0 in 0.046s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[36914b92-af04-45db-8300-7c45d76216bc]: (4, ('Logging out of session [sid: 53, target: iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb, portal: 172.16.0.220,3260]\nLogout of [sid: 53, target: iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 53, target: iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb, portal: 172.16.0.220,3260]
Apr 14 04:47:38 user nova-compute[71317]: Logout of [sid: 53, target: iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb, portal: 172.16.0.220,3260] successful.
Apr 14 04:47:38 user nova-compute[71317]:  stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op delete {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8e6a311-8ecf-435d-a4bd-23726576f9bb -p 172.16.0.220:3260 --op delete" returned: 0 in 0.017s {{(pid=81937) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec734804-39ca-4754-be38-fb108c8db6d2]: (4, ('', '')) {{(pid=81937) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71317) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.base [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.263s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG os_brick.initiator.connectors.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] <== disconnect_volume: return (264ms) None {{(pid=71317) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG nova.virt.libvirt.volume.iscsi [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Disconnected iSCSI Volume {{(pid=71317) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 04:47:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:39 user nova-compute[71317]: DEBUG nova.objects.instance [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'flavor' on Instance uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:47:39 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-51a5cbea-2d00-4679-881d-63f84c8ccba5 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "77f6bb83-54ab-485e-8943-742f2f576607" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.688s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:47:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:45 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:47:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:47:47 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:47:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:47:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:47:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:47:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:47:47 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updating instance_info_cache with network_info: [{"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:47:47 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-77f6bb83-54ab-485e-8943-742f2f576607" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:47:47 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:47:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:47:50 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:47:50 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:47:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:47:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:47:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:47:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:47:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:53 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:47:54 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:47:55 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:47:55 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8665MB free_disk=26.256359100341797GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance f5a130f4-02c3-41fc-bbdb-d2b844e1320b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 77f6bb83-54ab-485e-8943-742f2f576607 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=768MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing inventories for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating ProviderTree inventory for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating inventory in ProviderTree for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing aggregate associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, aggregates: None {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing trait associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, traits: COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSE41,HW_CPU_X86_SSE42,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_TRUSTED_CERTS,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_ACCELERATORS,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NET_VIF_MODEL_E1000,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_NODE,HW_CPU_X86_SSE2,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,HW_CPU_X86_SSSE3,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_STORAGE_BUS_USB,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_NE2K_PCI {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.457s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] There are 0 instances to clean {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:47:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:47:56 user nova-compute[71317]: INFO nova.compute.manager [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Terminating instance
Apr 14 04:47:56 user nova-compute[71317]: DEBUG nova.compute.manager [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-4de99f0e-7afb-4dea-ad20-931ceea897b0 req-b1166d20-0592-484f-90fb-796810d4d698 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Received event network-vif-unplugged-7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4de99f0e-7afb-4dea-ad20-931ceea897b0 req-b1166d20-0592-484f-90fb-796810d4d698 service nova] Acquiring lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4de99f0e-7afb-4dea-ad20-931ceea897b0 req-b1166d20-0592-484f-90fb-796810d4d698 service nova] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-4de99f0e-7afb-4dea-ad20-931ceea897b0 req-b1166d20-0592-484f-90fb-796810d4d698 service nova] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-4de99f0e-7afb-4dea-ad20-931ceea897b0 req-b1166d20-0592-484f-90fb-796810d4d698 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] No waiting events found dispatching network-vif-unplugged-7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG nova.compute.manager [req-4de99f0e-7afb-4dea-ad20-931ceea897b0 req-b1166d20-0592-484f-90fb-796810d4d698 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Received event network-vif-unplugged-7f00e811-6655-498b-a09b-5a45aba618b4 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Instance destroyed successfully.
Apr 14 04:47:56 user nova-compute[71317]: DEBUG nova.objects.instance [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lazy-loading 'resources' on Instance uuid f5a130f4-02c3-41fc-bbdb-d2b844e1320b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:45:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-441026141',display_name='tempest-AttachVolumeNegativeTest-server-441026141',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-441026141',id=46,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPq10jRsZwJAwjn8fWjAt5oHsMf2IPnRo2qmPAUD/9zrrvsoEgKc9ia04kVf6fotTh4Pv5yyx4MxgJSrLSbRN1rSGK9xMKoeETId4pSL+n09LaBGKTmlM0sgh6MwFAvp1A==',key_name='tempest-keypair-276811093',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:45:08Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='601e2806907a48d3995cc90b54da3c8c',ramdisk_id='',reservation_id='r-tcl7535t',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-1197484356',owner_user_name='tempest-AttachVolumeNegativeTest-1197484356-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:45:09Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7cab2461ab0d4eb08b62feaf6396bc87',uuid=f5a130f4-02c3-41fc-bbdb-d2b844e1320b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.114", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converting VIF {"id": "7f00e811-6655-498b-a09b-5a45aba618b4", "address": "fa:16:3e:1a:e9:5f", "network": {"id": "f58d1c12-4d40-40b9-93c5-2db82486289a", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-727207431-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.114", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "601e2806907a48d3995cc90b54da3c8c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f00e811-66", "ovs_interfaceid": "7f00e811-6655-498b-a09b-5a45aba618b4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:1a:e9:5f,bridge_name='br-int',has_traffic_filtering=True,id=7f00e811-6655-498b-a09b-5a45aba618b4,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f00e811-66') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG os_vif [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:1a:e9:5f,bridge_name='br-int',has_traffic_filtering=True,id=7f00e811-6655-498b-a09b-5a45aba618b4,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f00e811-66') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7f00e811-66, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:47:56 user nova-compute[71317]: INFO os_vif [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:1a:e9:5f,bridge_name='br-int',has_traffic_filtering=True,id=7f00e811-6655-498b-a09b-5a45aba618b4,network=Network(f58d1c12-4d40-40b9-93c5-2db82486289a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f00e811-66')
Apr 14 04:47:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Deleting instance files /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b_del
Apr 14 04:47:56 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Deletion of /opt/stack/data/nova/instances/f5a130f4-02c3-41fc-bbdb-d2b844e1320b_del complete
Apr 14 04:47:56 user nova-compute[71317]: INFO nova.compute.manager [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Took 0.84 seconds to destroy the instance on the hypervisor.
Apr 14 04:47:56 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:47:56 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:47:57 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:47:57 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Took 0.72 seconds to deallocate network for instance.
Apr 14 04:47:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:47:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:47:57 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:47:57 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:47:57 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:47:57 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:47:57 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:47:57 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:47:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.166s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:47:57 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Deleted allocations for instance f5a130f4-02c3-41fc-bbdb-d2b844e1320b
Apr 14 04:47:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-442642a7-fca0-4a92-adb4-972d55828a3a tempest-AttachVolumeNegativeTest-1197484356 tempest-AttachVolumeNegativeTest-1197484356-project-member] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.942s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:47:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-45518128-9334-4aa4-895f-f4b4a2371b52 req-9ee68d60-2f8d-4eb1-a2cd-0bac5bb12374 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Received event network-vif-plugged-7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:47:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-45518128-9334-4aa4-895f-f4b4a2371b52 req-9ee68d60-2f8d-4eb1-a2cd-0bac5bb12374 service nova] Acquiring lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:47:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-45518128-9334-4aa4-895f-f4b4a2371b52 req-9ee68d60-2f8d-4eb1-a2cd-0bac5bb12374 service nova] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:47:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-45518128-9334-4aa4-895f-f4b4a2371b52 req-9ee68d60-2f8d-4eb1-a2cd-0bac5bb12374 service nova] Lock "f5a130f4-02c3-41fc-bbdb-d2b844e1320b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:47:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-45518128-9334-4aa4-895f-f4b4a2371b52 req-9ee68d60-2f8d-4eb1-a2cd-0bac5bb12374 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] No waiting events found dispatching network-vif-plugged-7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:47:58 user nova-compute[71317]: WARNING nova.compute.manager [req-45518128-9334-4aa4-895f-f4b4a2371b52 req-9ee68d60-2f8d-4eb1-a2cd-0bac5bb12374 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Received unexpected event network-vif-plugged-7f00e811-6655-498b-a09b-5a45aba618b4 for instance with vm_state deleted and task_state None.
Apr 14 04:47:58 user nova-compute[71317]: DEBUG nova.compute.manager [req-45518128-9334-4aa4-895f-f4b4a2371b52 req-9ee68d60-2f8d-4eb1-a2cd-0bac5bb12374 service nova] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Received event network-vif-deleted-7f00e811-6655-498b-a09b-5a45aba618b4 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:48:01 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:06 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:48:11 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:48:11 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances with incomplete migration  {{(pid=71317) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 04:48:11 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447676.7406235, f5a130f4-02c3-41fc-bbdb-d2b844e1320b => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:48:11 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] VM Stopped (Lifecycle Event)
Apr 14 04:48:11 user nova-compute[71317]: DEBUG nova.compute.manager [None req-526df43b-fb9a-4a82-9ef1-fc2da7cc3712 None None] [instance: f5a130f4-02c3-41fc-bbdb-d2b844e1320b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:48:11 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:21 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "77f6bb83-54ab-485e-8943-742f2f576607" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:48:28 user nova-compute[71317]: INFO nova.compute.manager [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Terminating instance
Apr 14 04:48:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-dd9d7956-9837-44b3-b919-5f8fe2c07579 req-bb97c847-5f28-4c23-aa55-33027660deed service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-vif-unplugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-dd9d7956-9837-44b3-b919-5f8fe2c07579 req-bb97c847-5f28-4c23-aa55-33027660deed service nova] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-dd9d7956-9837-44b3-b919-5f8fe2c07579 req-bb97c847-5f28-4c23-aa55-33027660deed service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-dd9d7956-9837-44b3-b919-5f8fe2c07579 req-bb97c847-5f28-4c23-aa55-33027660deed service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-dd9d7956-9837-44b3-b919-5f8fe2c07579 req-bb97c847-5f28-4c23-aa55-33027660deed service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] No waiting events found dispatching network-vif-unplugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-dd9d7956-9837-44b3-b919-5f8fe2c07579 req-bb97c847-5f28-4c23-aa55-33027660deed service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-vif-unplugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:48:28 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Instance destroyed successfully.
Apr 14 04:48:28 user nova-compute[71317]: DEBUG nova.objects.instance [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'resources' on Instance uuid 77f6bb83-54ab-485e-8943-742f2f576607 {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:42:00Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1249800375',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1249800375',id=44,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH6fraox0ufDVlTeIxqf3LA1Jqk/6GB3uKQGBsTlrxVUqCfAkV+hrTL0aqpPNJ0Y4OrCsZ9AqcY4PXRY4CmEWIke2qyPR134zIiDdDnr3VeL889NFNXZb2Fua8pApWkvww==',key_name='tempest-keypair-1996810505',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:46:03Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c10c8cf348e04e44afbaae93a4b60e36',ramdisk_id='',reservation_id='r-4s0bcxm4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2141578557',owner_user_name='tempest-AttachVolumeShelveTestJSON-2141578557-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:46:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='67d6c699fc524554a6f4983af91d58fc',uuid=77f6bb83-54ab-485e-8943-742f2f576607,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converting VIF {"id": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "address": "fa:16:3e:6b:b2:cc", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap440e7ef0-4f", "ovs_interfaceid": "440e7ef0-4f4c-4923-ab57-8e1ba56dbea3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG os_vif [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap440e7ef0-4f, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:48:28 user nova-compute[71317]: INFO os_vif [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:6b:b2:cc,bridge_name='br-int',has_traffic_filtering=True,id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap440e7ef0-4f')
Apr 14 04:48:28 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Deleting instance files /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607_del
Apr 14 04:48:28 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Deletion of /opt/stack/data/nova/instances/77f6bb83-54ab-485e-8943-742f2f576607_del complete
Apr 14 04:48:29 user nova-compute[71317]: INFO nova.compute.manager [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Took 0.88 seconds to destroy the instance on the hypervisor.
Apr 14 04:48:29 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:48:29 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:48:29 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:48:29 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:48:29 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Took 0.71 seconds to deallocate network for instance.
Apr 14 04:48:29 user nova-compute[71317]: DEBUG nova.compute.manager [req-cd0aa771-bb46-4b4c-8a6e-df671151f3c5 req-afe73076-bfa3-4399-ab96-1bbe702d2f4c service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-vif-deleted-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:48:29 user nova-compute[71317]: INFO nova.compute.manager [req-cd0aa771-bb46-4b4c-8a6e-df671151f3c5 req-afe73076-bfa3-4399-ab96-1bbe702d2f4c service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Neutron deleted interface 440e7ef0-4f4c-4923-ab57-8e1ba56dbea3; detaching it from the instance and deleting it from the info cache
Apr 14 04:48:29 user nova-compute[71317]: DEBUG nova.network.neutron [req-cd0aa771-bb46-4b4c-8a6e-df671151f3c5 req-afe73076-bfa3-4399-ab96-1bbe702d2f4c service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:48:29 user nova-compute[71317]: DEBUG nova.compute.manager [req-cd0aa771-bb46-4b4c-8a6e-df671151f3c5 req-afe73076-bfa3-4399-ab96-1bbe702d2f4c service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Detach interface failed, port_id=440e7ef0-4f4c-4923-ab57-8e1ba56dbea3, reason: Instance 77f6bb83-54ab-485e-8943-742f2f576607 could not be found. {{(pid=71317) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 04:48:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:48:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:48:29 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:48:29 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:48:29 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.141s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:48:30 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Deleted allocations for instance 77f6bb83-54ab-485e-8943-742f2f576607
Apr 14 04:48:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-2ecb48f7-e1a1-414b-85f6-1d2fd82b127c tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "77f6bb83-54ab-485e-8943-742f2f576607" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.940s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:48:30 user nova-compute[71317]: DEBUG nova.compute.manager [req-c544242f-4f9d-497d-926c-dc763e8f1ae1 req-be813d30-3976-4c25-b9b5-3c7ab1fe756c service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received event network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:48:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c544242f-4f9d-497d-926c-dc763e8f1ae1 req-be813d30-3976-4c25-b9b5-3c7ab1fe756c service nova] Acquiring lock "77f6bb83-54ab-485e-8943-742f2f576607-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:48:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c544242f-4f9d-497d-926c-dc763e8f1ae1 req-be813d30-3976-4c25-b9b5-3c7ab1fe756c service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:48:30 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-c544242f-4f9d-497d-926c-dc763e8f1ae1 req-be813d30-3976-4c25-b9b5-3c7ab1fe756c service nova] Lock "77f6bb83-54ab-485e-8943-742f2f576607-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:48:30 user nova-compute[71317]: DEBUG nova.compute.manager [req-c544242f-4f9d-497d-926c-dc763e8f1ae1 req-be813d30-3976-4c25-b9b5-3c7ab1fe756c service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] No waiting events found dispatching network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:48:30 user nova-compute[71317]: WARNING nova.compute.manager [req-c544242f-4f9d-497d-926c-dc763e8f1ae1 req-be813d30-3976-4c25-b9b5-3c7ab1fe756c service nova] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Received unexpected event network-vif-plugged-440e7ef0-4f4c-4923-ab57-8e1ba56dbea3 for instance with vm_state deleted and task_state None.
Apr 14 04:48:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:43 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447708.94344, 77f6bb83-54ab-485e-8943-742f2f576607 => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:48:43 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] VM Stopped (Lifecycle Event)
Apr 14 04:48:43 user nova-compute[71317]: DEBUG nova.compute.manager [None req-acc31ae9-f39b-4e3a-8baf-ac779bb35807 None None] [instance: 77f6bb83-54ab-485e-8943-742f2f576607] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:48:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:48:46 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:48:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:48 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:48:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:48:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:48:48 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Didn't find any instances for network info cache update. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 04:48:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:48:52 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:48:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:48:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:48:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:48:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:48:53 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:48:55 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:48:55 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:48:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8839MB free_disk=26.294696807861328GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:48:55 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.149s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:48:56 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:48:57 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:48:57 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:48:58 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:48:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:48:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:04 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:49:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:49:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:49:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:49:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:49:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:49:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "068de634-b290-46ae-b5a2-ef36165cd23b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Starting instance... {{(pid=71317) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:49:22 user nova-compute[71317]: INFO nova.compute.claims [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Claim successful on node user
Apr 14 04:49:22 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.211s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Start building networks asynchronously for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Allocating IP information in the background. {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG nova.network.neutron [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] allocate_for_instance() {{(pid=71317) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 04:49:22 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 04:49:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Start building block device mappings for instance. {{(pid=71317) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG nova.policy [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '67d6c699fc524554a6f4983af91d58fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c10c8cf348e04e44afbaae93a4b60e36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71317) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG nova.compute.manager [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Start spawning the instance on the hypervisor. {{(pid=71317) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:49:22 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Creating image(s)
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "/opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "/opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "/opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "f78450b1f55085246e23ff0f51529f31556ca16d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d,backing_fmt=raw /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk 1073741824" returned: 0 in 0.054s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "f78450b1f55085246e23ff0f51529f31556ca16d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.196s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:49:22 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Checking if we can resize image /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk. size=1073741824 {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG nova.network.neutron [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Successfully created port: 2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG nova.virt.disk.api [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Cannot resize image /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk to a smaller size. {{(pid=71317) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG nova.objects.instance [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'migration_context' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Ensure instance console log exists: /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:49:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.network.neutron [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Successfully updated port: 2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquired lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.network.neutron [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.network.neutron [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Instance cache missing network info. {{(pid=71317) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.compute.manager [req-b3b9b306-f3b0-4f1e-8bdf-89d7bdafab37 req-440e5c88-b97f-4737-a94e-65cfc04d9071 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-changed-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.compute.manager [req-b3b9b306-f3b0-4f1e-8bdf-89d7bdafab37 req-440e5c88-b97f-4737-a94e-65cfc04d9071 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Refreshing instance network info cache due to event network-changed-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b3b9b306-f3b0-4f1e-8bdf-89d7bdafab37 req-440e5c88-b97f-4737-a94e-65cfc04d9071 service nova] Acquiring lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.network.neutron [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating instance_info_cache with network_info: [{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Releasing lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.compute.manager [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Instance network_info: |[{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71317) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b3b9b306-f3b0-4f1e-8bdf-89d7bdafab37 req-440e5c88-b97f-4737-a94e-65cfc04d9071 service nova] Acquired lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.network.neutron [req-b3b9b306-f3b0-4f1e-8bdf-89d7bdafab37 req-440e5c88-b97f-4737-a94e-65cfc04d9071 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Refreshing network info cache for port 2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Start _get_guest_xml network_info=[{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:49:24 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:49:24 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T04:12:47Z,direct_url=<?>,disk_format='qcow2',id=5c55a1da-f6f8-4442-939b-f0cb72ea4327,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='d8b3c3fd68cb4219826c2a6fd8b21939',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T04:12:48Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:49:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-203487374',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-203487374',id=47,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIz9kPiXrU+d9YNNeLmDz42lc/lqPHsJkMzNHXLJ9BxaT3PcpDfhDBMRUsgkcRE+o41EINj1zxZFQZUWhpX4XIsgWigDmt6Wb2qr1dlIExZrdUiB9GwXx8T2hNCZNX55JA==',key_name='tempest-keypair-1115532707',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c10c8cf348e04e44afbaae93a4b60e36',ramdisk_id='',reservation_id='r-p7q5eq40',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-2141578557',owner_user_name='tempest-AttachVolumeShelveTestJSON-2141578557-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:49:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='67d6c699fc524554a6f4983af91d58fc',uuid=068de634-b290-46ae-b5a2-ef36165cd23b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converting VIF {"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.objects.instance [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'pci_devices' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:49:24 user nova-compute[71317]:   <uuid>068de634-b290-46ae-b5a2-ef36165cd23b</uuid>
Apr 14 04:49:24 user nova-compute[71317]:   <name>instance-0000002f</name>
Apr 14 04:49:24 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:49:24 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:49:24 user nova-compute[71317]:   <metadata>
Apr 14 04:49:24 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:49:24 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:49:24 user nova-compute[71317]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-203487374</nova:name>
Apr 14 04:49:24 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:49:24</nova:creationTime>
Apr 14 04:49:24 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:49:24 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:49:24 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:49:24 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:49:24 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:49:24 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:49:24 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:49:24 user nova-compute[71317]:       <nova:owner>
Apr 14 04:49:24 user nova-compute[71317]:         <nova:user uuid="67d6c699fc524554a6f4983af91d58fc">tempest-AttachVolumeShelveTestJSON-2141578557-project-member</nova:user>
Apr 14 04:49:24 user nova-compute[71317]:         <nova:project uuid="c10c8cf348e04e44afbaae93a4b60e36">tempest-AttachVolumeShelveTestJSON-2141578557</nova:project>
Apr 14 04:49:24 user nova-compute[71317]:       </nova:owner>
Apr 14 04:49:24 user nova-compute[71317]:       <nova:root type="image" uuid="5c55a1da-f6f8-4442-939b-f0cb72ea4327"/>
Apr 14 04:49:24 user nova-compute[71317]:       <nova:ports>
Apr 14 04:49:24 user nova-compute[71317]:         <nova:port uuid="2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b">
Apr 14 04:49:24 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 14 04:49:24 user nova-compute[71317]:         </nova:port>
Apr 14 04:49:24 user nova-compute[71317]:       </nova:ports>
Apr 14 04:49:24 user nova-compute[71317]:     </nova:instance>
Apr 14 04:49:24 user nova-compute[71317]:   </metadata>
Apr 14 04:49:24 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:49:24 user nova-compute[71317]:     <system>
Apr 14 04:49:24 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:49:24 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:49:24 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:49:24 user nova-compute[71317]:       <entry name="serial">068de634-b290-46ae-b5a2-ef36165cd23b</entry>
Apr 14 04:49:24 user nova-compute[71317]:       <entry name="uuid">068de634-b290-46ae-b5a2-ef36165cd23b</entry>
Apr 14 04:49:24 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:49:24 user nova-compute[71317]:     </system>
Apr 14 04:49:24 user nova-compute[71317]:   </sysinfo>
Apr 14 04:49:24 user nova-compute[71317]:   <os>
Apr 14 04:49:24 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:49:24 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:49:24 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:49:24 user nova-compute[71317]:   </os>
Apr 14 04:49:24 user nova-compute[71317]:   <features>
Apr 14 04:49:24 user nova-compute[71317]:     <acpi/>
Apr 14 04:49:24 user nova-compute[71317]:     <apic/>
Apr 14 04:49:24 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:49:24 user nova-compute[71317]:   </features>
Apr 14 04:49:24 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:49:24 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:49:24 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:49:24 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:49:24 user nova-compute[71317]:   </clock>
Apr 14 04:49:24 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:49:24 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:49:24 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:49:24 user nova-compute[71317]:   </cpu>
Apr 14 04:49:24 user nova-compute[71317]:   <devices>
Apr 14 04:49:24 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:49:24 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:49:24 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk"/>
Apr 14 04:49:24 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:49:24 user nova-compute[71317]:     </disk>
Apr 14 04:49:24 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:49:24 user nova-compute[71317]:       <mac address="fa:16:3e:65:0b:60"/>
Apr 14 04:49:24 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:49:24 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:49:24 user nova-compute[71317]:       <target dev="tap2cd60678-2b"/>
Apr 14 04:49:24 user nova-compute[71317]:     </interface>
Apr 14 04:49:24 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:49:24 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/console.log" append="off"/>
Apr 14 04:49:24 user nova-compute[71317]:     </serial>
Apr 14 04:49:24 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:49:24 user nova-compute[71317]:     <video>
Apr 14 04:49:24 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:49:24 user nova-compute[71317]:     </video>
Apr 14 04:49:24 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:49:24 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:49:24 user nova-compute[71317]:     </rng>
Apr 14 04:49:24 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:49:24 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:49:24 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:49:24 user nova-compute[71317]:     </memballoon>
Apr 14 04:49:24 user nova-compute[71317]:   </devices>
Apr 14 04:49:24 user nova-compute[71317]: </domain>
Apr 14 04:49:24 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:49:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-203487374',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-203487374',id=47,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIz9kPiXrU+d9YNNeLmDz42lc/lqPHsJkMzNHXLJ9BxaT3PcpDfhDBMRUsgkcRE+o41EINj1zxZFQZUWhpX4XIsgWigDmt6Wb2qr1dlIExZrdUiB9GwXx8T2hNCZNX55JA==',key_name='tempest-keypair-1115532707',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c10c8cf348e04e44afbaae93a4b60e36',ramdisk_id='',reservation_id='r-p7q5eq40',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-2141578557',owner_user_name='tempest-AttachVolumeShelveTestJSON-2141578557-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:49:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='67d6c699fc524554a6f4983af91d58fc',uuid=068de634-b290-46ae-b5a2-ef36165cd23b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converting VIF {"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG os_vif [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2cd60678-2b, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2cd60678-2b, col_values=(('external_ids', {'iface-id': '2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:65:0b:60', 'vm-uuid': '068de634-b290-46ae-b5a2-ef36165cd23b'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:24 user nova-compute[71317]: INFO os_vif [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b')
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] No VIF found with MAC fa:16:3e:65:0b:60, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.network.neutron [req-b3b9b306-f3b0-4f1e-8bdf-89d7bdafab37 req-440e5c88-b97f-4737-a94e-65cfc04d9071 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updated VIF entry in instance network info cache for port 2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG nova.network.neutron [req-b3b9b306-f3b0-4f1e-8bdf-89d7bdafab37 req-440e5c88-b97f-4737-a94e-65cfc04d9071 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating instance_info_cache with network_info: [{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:49:24 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-b3b9b306-f3b0-4f1e-8bdf-89d7bdafab37 req-440e5c88-b97f-4737-a94e-65cfc04d9071 service nova] Releasing lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:49:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-a55a28e2-7a55-4a8d-94a1-fe2f3ae35b13 req-93b6ec7b-be68-4226-a5fb-79e60743d61e service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:49:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a55a28e2-7a55-4a8d-94a1-fe2f3ae35b13 req-93b6ec7b-be68-4226-a5fb-79e60743d61e service nova] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:49:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a55a28e2-7a55-4a8d-94a1-fe2f3ae35b13 req-93b6ec7b-be68-4226-a5fb-79e60743d61e service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:49:26 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-a55a28e2-7a55-4a8d-94a1-fe2f3ae35b13 req-93b6ec7b-be68-4226-a5fb-79e60743d61e service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:49:26 user nova-compute[71317]: DEBUG nova.compute.manager [req-a55a28e2-7a55-4a8d-94a1-fe2f3ae35b13 req-93b6ec7b-be68-4226-a5fb-79e60743d61e service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] No waiting events found dispatching network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:49:26 user nova-compute[71317]: WARNING nova.compute.manager [req-a55a28e2-7a55-4a8d-94a1-fe2f3ae35b13 req-93b6ec7b-be68-4226-a5fb-79e60743d61e service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received unexpected event network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b for instance with vm_state building and task_state spawning.
Apr 14 04:49:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:26 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447768.134032, 068de634-b290-46ae-b5a2-ef36165cd23b => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:49:28 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] VM Resumed (Lifecycle Event)
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:49:28 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Instance spawned successfully.
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Found default for hw_cdrom_bus of ide {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Found default for hw_disk_bus of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Found default for hw_input_bus of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Found default for hw_pointer_model of None {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Found default for hw_video_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Found default for hw_vif_model of virtio {{(pid=71317) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 04:49:28 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447768.1374707, 068de634-b290-46ae-b5a2-ef36165cd23b => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:49:28 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] VM Started (Lifecycle Event)
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:49:28 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:49:28 user nova-compute[71317]: INFO nova.compute.manager [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Took 5.65 seconds to spawn the instance on the hypervisor.
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.compute.manager [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-6afab1d7-5377-4ce6-bd84-7552be8bf523 req-d812bff4-b363-4eba-a9a0-c08771b942d0 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6afab1d7-5377-4ce6-bd84-7552be8bf523 req-d812bff4-b363-4eba-a9a0-c08771b942d0 service nova] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6afab1d7-5377-4ce6-bd84-7552be8bf523 req-d812bff4-b363-4eba-a9a0-c08771b942d0 service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6afab1d7-5377-4ce6-bd84-7552be8bf523 req-d812bff4-b363-4eba-a9a0-c08771b942d0 service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG nova.compute.manager [req-6afab1d7-5377-4ce6-bd84-7552be8bf523 req-d812bff4-b363-4eba-a9a0-c08771b942d0 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] No waiting events found dispatching network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:49:28 user nova-compute[71317]: WARNING nova.compute.manager [req-6afab1d7-5377-4ce6-bd84-7552be8bf523 req-d812bff4-b363-4eba-a9a0-c08771b942d0 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received unexpected event network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b for instance with vm_state building and task_state spawning.
Apr 14 04:49:28 user nova-compute[71317]: INFO nova.compute.manager [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Took 6.23 seconds to build instance.
Apr 14 04:49:28 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9328b939-a1bb-4d47-8c17-e1bb683b347b tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "068de634-b290-46ae-b5a2-ef36165cd23b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.334s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:49:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:49:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:49:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:48 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:49:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:49:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:49:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:49:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:49:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:49:49 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:49:49 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lazy-loading 'info_cache' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:49:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:49 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating instance_info_cache with network_info: [{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:49:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:49:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:49:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:49:52 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:49:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:49:56 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:49:57 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:49:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:49:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:49:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:49:57 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:49:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:49:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:49:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:49:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:49:57 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:49:57 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:49:57 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8758MB free_disk=26.271236419677734GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:49:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:49:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:49:58 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 068de634-b290-46ae-b5a2-ef36165cd23b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:49:58 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:49:58 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:49:58 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:49:58 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:49:58 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:49:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.233s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:49:58 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:49:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:49:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:49:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:49:59 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:49:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:50:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:50:08 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:50:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:50:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:50:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:50:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4998-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:50:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:50:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:50:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:50:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:50:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:50:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:50:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:50:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:50:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:50:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:50:24 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:50:29 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:50:34 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:50:39 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:50:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:50:44 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:50:48 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:50:49 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:50:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:50:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:50:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:50:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:50:49 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:50:49 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lazy-loading 'info_cache' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:50:49 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:50:49 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating instance_info_cache with network_info: [{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:50:49 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:50:49 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:50:51 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:50:52 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:50:54 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:50:57 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:50:57 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:50:57 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8730MB free_disk=26.26102066040039GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 068de634-b290-46ae-b5a2-ef36165cd23b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:50:57 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:50:58 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:50:58 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:50:58 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:50:58 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.166s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:50:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:50:59 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:50:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:50:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:50:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:09 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:51:09 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-ad58e192-328a-4276-ba1e-6c345ddd275a req-d2f7ea21-0a2b-425e-9206-f5f845c41e1f service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-changed-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:51:13 user nova-compute[71317]: DEBUG nova.compute.manager [req-ad58e192-328a-4276-ba1e-6c345ddd275a req-d2f7ea21-0a2b-425e-9206-f5f845c41e1f service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Refreshing instance network info cache due to event network-changed-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:51:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ad58e192-328a-4276-ba1e-6c345ddd275a req-d2f7ea21-0a2b-425e-9206-f5f845c41e1f service nova] Acquiring lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:51:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ad58e192-328a-4276-ba1e-6c345ddd275a req-d2f7ea21-0a2b-425e-9206-f5f845c41e1f service nova] Acquired lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:51:13 user nova-compute[71317]: DEBUG nova.network.neutron [req-ad58e192-328a-4276-ba1e-6c345ddd275a req-d2f7ea21-0a2b-425e-9206-f5f845c41e1f service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Refreshing network info cache for port 2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:51:13 user nova-compute[71317]: DEBUG nova.network.neutron [req-ad58e192-328a-4276-ba1e-6c345ddd275a req-d2f7ea21-0a2b-425e-9206-f5f845c41e1f service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updated VIF entry in instance network info cache for port 2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:51:13 user nova-compute[71317]: DEBUG nova.network.neutron [req-ad58e192-328a-4276-ba1e-6c345ddd275a req-d2f7ea21-0a2b-425e-9206-f5f845c41e1f service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating instance_info_cache with network_info: [{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:51:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-ad58e192-328a-4276-ba1e-6c345ddd275a req-d2f7ea21-0a2b-425e-9206-f5f845c41e1f service nova] Releasing lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:51:13 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:14 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:51:16 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "068de634-b290-46ae-b5a2-ef36165cd23b" acquired by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:51:16 user nova-compute[71317]: INFO nova.compute.manager [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Shelving
Apr 14 04:51:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-36a756dd-ce6e-4911-95c0-08b86a401b27 req-59d2698c-6caf-4bed-95ad-dbc1fe18e421 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-vif-unplugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:51:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36a756dd-ce6e-4911-95c0-08b86a401b27 req-59d2698c-6caf-4bed-95ad-dbc1fe18e421 service nova] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:51:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36a756dd-ce6e-4911-95c0-08b86a401b27 req-59d2698c-6caf-4bed-95ad-dbc1fe18e421 service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:51:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-36a756dd-ce6e-4911-95c0-08b86a401b27 req-59d2698c-6caf-4bed-95ad-dbc1fe18e421 service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:51:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-36a756dd-ce6e-4911-95c0-08b86a401b27 req-59d2698c-6caf-4bed-95ad-dbc1fe18e421 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] No waiting events found dispatching network-vif-unplugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:51:17 user nova-compute[71317]: WARNING nova.compute.manager [req-36a756dd-ce6e-4911-95c0-08b86a401b27 req-59d2698c-6caf-4bed-95ad-dbc1fe18e421 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received unexpected event network-vif-unplugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b for instance with vm_state active and task_state shelving.
Apr 14 04:51:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Instance destroyed successfully.
Apr 14 04:51:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:17 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Beginning cold snapshot process
Apr 14 04:51:17 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:51:17 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk /opt/stack/data/nova/instances/snapshots/tmpikkplhkn/91e660064cb443bdabe9fd7c7abdb47f {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:51:18 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk /opt/stack/data/nova/instances/snapshots/tmpikkplhkn/91e660064cb443bdabe9fd7c7abdb47f" returned: 0 in 0.600s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:51:18 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Snapshot extracted, beginning image upload
Apr 14 04:51:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-6df4f61d-bddb-45a5-8e32-d920377b0680 req-7bf1f743-df17-4039-bcb4-fa1c4e695a1d service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:51:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6df4f61d-bddb-45a5-8e32-d920377b0680 req-7bf1f743-df17-4039-bcb4-fa1c4e695a1d service nova] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:51:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6df4f61d-bddb-45a5-8e32-d920377b0680 req-7bf1f743-df17-4039-bcb4-fa1c4e695a1d service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:51:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6df4f61d-bddb-45a5-8e32-d920377b0680 req-7bf1f743-df17-4039-bcb4-fa1c4e695a1d service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:51:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-6df4f61d-bddb-45a5-8e32-d920377b0680 req-7bf1f743-df17-4039-bcb4-fa1c4e695a1d service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] No waiting events found dispatching network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:51:19 user nova-compute[71317]: WARNING nova.compute.manager [req-6df4f61d-bddb-45a5-8e32-d920377b0680 req-7bf1f743-df17-4039-bcb4-fa1c4e695a1d service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received unexpected event network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b for instance with vm_state active and task_state shelving_image_uploading.
Apr 14 04:51:19 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:20 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Snapshot image upload complete
Apr 14 04:51:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:51:20 user nova-compute[71317]: INFO nova.compute.manager [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Shelve offloading
Apr 14 04:51:20 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Instance destroyed successfully.
Apr 14 04:51:20 user nova-compute[71317]: DEBUG nova.compute.manager [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:51:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:51:20 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquired lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:51:20 user nova-compute[71317]: DEBUG nova.network.neutron [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:51:21 user nova-compute[71317]: DEBUG nova.network.neutron [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating instance_info_cache with network_info: [{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:51:21 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Releasing lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:51:21 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Instance destroyed successfully.
Apr 14 04:51:21 user nova-compute[71317]: DEBUG nova.objects.instance [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'resources' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:51:21 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T04:49:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-203487374',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-203487374',id=47,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIz9kPiXrU+d9YNNeLmDz42lc/lqPHsJkMzNHXLJ9BxaT3PcpDfhDBMRUsgkcRE+o41EINj1zxZFQZUWhpX4XIsgWigDmt6Wb2qr1dlIExZrdUiB9GwXx8T2hNCZNX55JA==',key_name='tempest-keypair-1115532707',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:49:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='c10c8cf348e04e44afbaae93a4b60e36',ramdisk_id='',reservation_id='r-p7q5eq40',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2141578557',owner_user_name='tempest-AttachVolumeShelveTestJSON-2141578557-project-member',shelved_at='2023-04-14T04:51:20.505352',shelved_host='user',shelved_image_id='840c6f79-54b7-4b33-a18f-64cffbd19799'},tags=<?>,task_state='shelving_offloading',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:51:18Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='67d6c699fc524554a6f4983af91d58fc',uuid=068de634-b290-46ae-b5a2-ef36165cd23b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='shelved') vif={"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:51:21 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converting VIF {"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:51:21 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:51:21 user nova-compute[71317]: DEBUG os_vif [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-daa4f8c1-4457-4892-ba44-ca0699d29715 req-e3c0f237-f7d3-4c14-a76f-f11b9f293b35 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-changed-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG nova.compute.manager [req-daa4f8c1-4457-4892-ba44-ca0699d29715 req-e3c0f237-f7d3-4c14-a76f-f11b9f293b35 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Refreshing instance network info cache due to event network-changed-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-daa4f8c1-4457-4892-ba44-ca0699d29715 req-e3c0f237-f7d3-4c14-a76f-f11b9f293b35 service nova] Acquiring lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-daa4f8c1-4457-4892-ba44-ca0699d29715 req-e3c0f237-f7d3-4c14-a76f-f11b9f293b35 service nova] Acquired lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG nova.network.neutron [req-daa4f8c1-4457-4892-ba44-ca0699d29715 req-e3c0f237-f7d3-4c14-a76f-f11b9f293b35 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Refreshing network info cache for port 2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2cd60678-2b, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:51:22 user nova-compute[71317]: INFO os_vif [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b')
Apr 14 04:51:22 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Deleting instance files /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b_del
Apr 14 04:51:22 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Deletion of /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b_del complete
Apr 14 04:51:22 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Deleted allocations for instance 068de634-b290-46ae-b5a2-ef36165cd23b
Apr 14 04:51:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG nova.objects.instance [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'numa_topology' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.104s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-9c0495b0-3f51-4c38-9781-0e79bd5f2c7e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "068de634-b290-46ae-b5a2-ef36165cd23b" "released" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: held 5.693s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG nova.network.neutron [req-daa4f8c1-4457-4892-ba44-ca0699d29715 req-e3c0f237-f7d3-4c14-a76f-f11b9f293b35 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updated VIF entry in instance network info cache for port 2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG nova.network.neutron [req-daa4f8c1-4457-4892-ba44-ca0699d29715 req-e3c0f237-f7d3-4c14-a76f-f11b9f293b35 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating instance_info_cache with network_info: [{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tap2cd60678-2b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:51:22 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-daa4f8c1-4457-4892-ba44-ca0699d29715 req-e3c0f237-f7d3-4c14-a76f-f11b9f293b35 service nova] Releasing lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:51:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:28 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:32 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681447877.2704144, 068de634-b290-46ae-b5a2-ef36165cd23b => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:51:32 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] VM Stopped (Lifecycle Event)
Apr 14 04:51:32 user nova-compute[71317]: DEBUG nova.compute.manager [None req-8fe7d45b-6142-4fb1-9441-4f6d3eafd682 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:51:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:51:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:43 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:48 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:51:50 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:51:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:51:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:51:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Didn't find any instances for network info cache update. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 04:51:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:52 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:51:53 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:51:57 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:51:57 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:51:57 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8880MB free_disk=26.242687225341797GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:51:57 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.138s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:51:58 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:51:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:51:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:51:59 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:52:00 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:52:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:52:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:52:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:52:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:52:18 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._run_image_cache_manager_pass {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:52:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "storage-registry-lock" by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:52:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:52:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "storage-registry-lock" "released" by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:52:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "storage-registry-lock" by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:52:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:52:18 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "storage-registry-lock" "released" by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:52:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Verify base images {{(pid=71317) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:314}}
Apr 14 04:52:18 user nova-compute[71317]: WARNING nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Unknown base file: /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e
Apr 14 04:52:18 user nova-compute[71317]: WARNING nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Unknown base file: /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9
Apr 14 04:52:18 user nova-compute[71317]: WARNING nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Unknown base file: /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af
Apr 14 04:52:18 user nova-compute[71317]: WARNING nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Unknown base file: /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91
Apr 14 04:52:18 user nova-compute[71317]: WARNING nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Unknown base file: /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592
Apr 14 04:52:18 user nova-compute[71317]: WARNING nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Unknown base file: /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812
Apr 14 04:52:18 user nova-compute[71317]: WARNING nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Unknown base file: /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf
Apr 14 04:52:18 user nova-compute[71317]: WARNING nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Unknown base file: /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658
Apr 14 04:52:18 user nova-compute[71317]: WARNING nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Unknown base file: /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d
Apr 14 04:52:18 user nova-compute[71317]: INFO nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Removable base files: /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9 /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91 /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592 /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812 /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658 /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d
Apr 14 04:52:18 user nova-compute[71317]: INFO nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/be368c247365fec2f9d8f8c8897c09b99127635e
Apr 14 04:52:18 user nova-compute[71317]: INFO nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/b316d97aa3eaa73df897c497fcec164014ef27e9
Apr 14 04:52:18 user nova-compute[71317]: INFO nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/aafe06d5ca392c133765c9fd1ef5f8d77b20d6af
Apr 14 04:52:18 user nova-compute[71317]: INFO nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/c262e1265bd42d036e70ac0a04884f55c3aa3f91
Apr 14 04:52:18 user nova-compute[71317]: INFO nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/76dfe33fcdd199f313fccbe5149d862d0c92a592
Apr 14 04:52:18 user nova-compute[71317]: INFO nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/7a87dc15b3f1a9b23f4facddd2d5d74578249812
Apr 14 04:52:18 user nova-compute[71317]: INFO nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/6f607dc5a06f489ce090518d55d9498f3378cadf
Apr 14 04:52:18 user nova-compute[71317]: INFO nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/8ac6af1c271197b8c863a86553be324e1abbd658
Apr 14 04:52:18 user nova-compute[71317]: INFO nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d
Apr 14 04:52:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Verification complete {{(pid=71317) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:350}}
Apr 14 04:52:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Verify swap images {{(pid=71317) _age_and_verify_swap_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:299}}
Apr 14 04:52:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.imagecache [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Verify ephemeral images {{(pid=71317) _age_and_verify_ephemeral_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:284}}
Apr 14 04:52:22 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:52:27 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:52:32 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:52:37 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:52:42 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:52:47 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:52:48 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:52:50 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:52:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:52:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:52:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Didn't find any instances for network info cache update. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 04:52:52 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:52:52 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:52:54 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:52:57 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:52:58 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:52:59 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:52:59 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8891MB free_disk=26.242111206054688GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing inventories for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating ProviderTree inventory for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Updating inventory in ProviderTree for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing aggregate associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, aggregates: None {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Refreshing trait associations for resource provider e35a99e4-b1ea-4792-a9ff-f93045b2018b, traits: COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSE41,HW_CPU_X86_SSE42,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_TRUSTED_CERTS,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_ACCELERATORS,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NET_VIF_MODEL_E1000,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_NODE,HW_CPU_X86_SSE2,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,HW_CPU_X86_SSSE3,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_STORAGE_BUS_USB,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_NE2K_PCI {{(pid=71317) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:52:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.400s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:53:01 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:01 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:02 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:53:02 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:07 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:08 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 04:53:08 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] There are 0 instances to clean {{(pid=71317) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 04:53:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:53:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "068de634-b290-46ae-b5a2-ef36165cd23b" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:53:09 user nova-compute[71317]: INFO nova.compute.manager [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Unshelving
Apr 14 04:53:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:53:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:53:09 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'pci_requests' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:53:09 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'numa_topology' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:53:09 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71317) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 04:53:09 user nova-compute[71317]: INFO nova.compute.claims [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Claim successful on node user
Apr 14 04:53:09 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:53:09 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:53:09 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.212s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:53:09 user nova-compute[71317]: INFO nova.network.neutron [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating port 2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Apr 14 04:53:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:53:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquired lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:53:10 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Building network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 04:53:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-667a040a-15eb-493c-9e9f-0ae8df92cddb req-ae64a96d-b84b-4f32-b6fc-fa4c35318365 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-changed-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:53:10 user nova-compute[71317]: DEBUG nova.compute.manager [req-667a040a-15eb-493c-9e9f-0ae8df92cddb req-ae64a96d-b84b-4f32-b6fc-fa4c35318365 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Refreshing instance network info cache due to event network-changed-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b. {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 04:53:10 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-667a040a-15eb-493c-9e9f-0ae8df92cddb req-ae64a96d-b84b-4f32-b6fc-fa4c35318365 service nova] Acquiring lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:53:11 user nova-compute[71317]: DEBUG nova.network.neutron [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating instance_info_cache with network_info: [{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:53:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Releasing lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:53:11 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Creating instance directory {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 04:53:11 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Creating image(s)
Apr 14 04:53:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "/opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:53:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "/opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:53:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "/opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.014s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:53:11 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'trusted_certs' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:53:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-667a040a-15eb-493c-9e9f-0ae8df92cddb req-ae64a96d-b84b-4f32-b6fc-fa4c35318365 service nova] Acquired lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:53:11 user nova-compute[71317]: DEBUG nova.network.neutron [req-667a040a-15eb-493c-9e9f-0ae8df92cddb req-ae64a96d-b84b-4f32-b6fc-fa4c35318365 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Refreshing network info cache for port 2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 04:53:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:53:11 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:53:11 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5.part --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5.part --force-share --output=json" returned: 0 in 0.136s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG nova.virt.images [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] 840c6f79-54b7-4b33-a18f-64cffbd19799 was qcow2, converting to raw {{(pid=71317) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG nova.privsep.utils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71317) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5.part /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5.converted {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG nova.network.neutron [req-667a040a-15eb-493c-9e9f-0ae8df92cddb req-ae64a96d-b84b-4f32-b6fc-fa4c35318365 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updated VIF entry in instance network info cache for port 2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b. {{(pid=71317) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG nova.network.neutron [req-667a040a-15eb-493c-9e9f-0ae8df92cddb req-ae64a96d-b84b-4f32-b6fc-fa4c35318365 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating instance_info_cache with network_info: [{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-667a040a-15eb-493c-9e9f-0ae8df92cddb req-ae64a96d-b84b-4f32-b6fc-fa4c35318365 service nova] Releasing lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5.part /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5.converted" returned: 0 in 0.479s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5.converted --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5.converted --force-share --output=json" returned: 0 in 0.135s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.657s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5 --force-share --output=json" returned: 0 in 0.133s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:53:12 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:53:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5 --force-share --output=json" returned: 0 in 0.130s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:53:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5,backing_fmt=raw /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk 1073741824 {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:53:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5,backing_fmt=raw /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk 1073741824" returned: 0 in 0.052s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:53:13 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.189s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:53:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5 --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:53:13 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/fde0eb6ead0ea815040dcef22a8f6e0398d6c8f5 --force-share --output=json" returned: 0 in 0.141s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:53:13 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'migration_context' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:53:13 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Rebasing disk image.
Apr 14 04:53:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:53:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d --force-share --output=json" returned: 0 in 0.129s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:53:13 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Running cmd (subprocess): qemu-img rebase -b /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d -F raw /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CMD "qemu-img rebase -b /opt/stack/data/nova/instances/_base/f78450b1f55085246e23ff0f51529f31556ca16d -F raw /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk" returned: 0 in 1.976s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Created local disks {{(pid=71317) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Ensure instance console log exists: /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/console.log {{(pid=71317) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Start _get_guest_xml network_info=[{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='37a5abb78bab1dc2aa295e4e2b20074a',container_format='bare',created_at=2023-04-14T04:51:16Z,direct_url=<?>,disk_format='qcow2',id=840c6f79-54b7-4b33-a18f-64cffbd19799,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-203487374-shelved',owner='c10c8cf348e04e44afbaae93a4b60e36',properties=ImageMetaProps,protected=<?>,size=38600704,status='active',tags=<?>,updated_at=2023-04-14T04:51:20Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'boot_index': 0, 'encrypted': False, 'guest_format': None, 'device_type': 'disk', 'size': 0, 'encryption_format': None, 'encryption_secret_uuid': None, 'encryption_options': None, 'device_name': '/dev/vda', 'image_id': '5c55a1da-f6f8-4442-939b-f0cb72ea4327'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 04:53:15 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:53:15 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71317) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T04:13:53Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='37a5abb78bab1dc2aa295e4e2b20074a',container_format='bare',created_at=2023-04-14T04:51:16Z,direct_url=<?>,disk_format='qcow2',id=840c6f79-54b7-4b33-a18f-64cffbd19799,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-203487374-shelved',owner='c10c8cf348e04e44afbaae93a4b60e36',properties=ImageMetaProps,protected=<?>,size=38600704,status='active',tags=<?>,updated_at=2023-04-14T04:51:20Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Flavor limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Image limits 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Flavor pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Image pref 0:0:0 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71317) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Got 1 possible topologies {{(pid=71317) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.hardware [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71317) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'vcpu_model' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:49:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-203487374',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-203487374',id=47,image_ref='840c6f79-54b7-4b33-a18f-64cffbd19799',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-keypair-1115532707',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:49:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='c10c8cf348e04e44afbaae93a4b60e36',ramdisk_id='',reservation_id='r-p7q5eq40',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2141578557',owner_user_name='tempest-AttachVolumeShelveTestJSON-2141578557-project-member',shelved_at='2023-04-14T04:51:20.505352',shelved_host='user',shelved_image_id='840c6f79-54b7-4b33-a18f-64cffbd19799'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:53:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='67d6c699fc524554a6f4983af91d58fc',uuid=068de634-b290-46ae-b5a2-ef36165cd23b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71317) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converting VIF {"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.objects.instance [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'pci_devices' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] End _get_guest_xml xml=<domain type="kvm">
Apr 14 04:53:15 user nova-compute[71317]:   <uuid>068de634-b290-46ae-b5a2-ef36165cd23b</uuid>
Apr 14 04:53:15 user nova-compute[71317]:   <name>instance-0000002f</name>
Apr 14 04:53:15 user nova-compute[71317]:   <memory>131072</memory>
Apr 14 04:53:15 user nova-compute[71317]:   <vcpu>1</vcpu>
Apr 14 04:53:15 user nova-compute[71317]:   <metadata>
Apr 14 04:53:15 user nova-compute[71317]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 04:53:15 user nova-compute[71317]:       <nova:package version="0.0.0"/>
Apr 14 04:53:15 user nova-compute[71317]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-203487374</nova:name>
Apr 14 04:53:15 user nova-compute[71317]:       <nova:creationTime>2023-04-14 04:53:15</nova:creationTime>
Apr 14 04:53:15 user nova-compute[71317]:       <nova:flavor name="m1.nano">
Apr 14 04:53:15 user nova-compute[71317]:         <nova:memory>128</nova:memory>
Apr 14 04:53:15 user nova-compute[71317]:         <nova:disk>1</nova:disk>
Apr 14 04:53:15 user nova-compute[71317]:         <nova:swap>0</nova:swap>
Apr 14 04:53:15 user nova-compute[71317]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 04:53:15 user nova-compute[71317]:         <nova:vcpus>1</nova:vcpus>
Apr 14 04:53:15 user nova-compute[71317]:       </nova:flavor>
Apr 14 04:53:15 user nova-compute[71317]:       <nova:owner>
Apr 14 04:53:15 user nova-compute[71317]:         <nova:user uuid="67d6c699fc524554a6f4983af91d58fc">tempest-AttachVolumeShelveTestJSON-2141578557-project-member</nova:user>
Apr 14 04:53:15 user nova-compute[71317]:         <nova:project uuid="c10c8cf348e04e44afbaae93a4b60e36">tempest-AttachVolumeShelveTestJSON-2141578557</nova:project>
Apr 14 04:53:15 user nova-compute[71317]:       </nova:owner>
Apr 14 04:53:15 user nova-compute[71317]:       <nova:root type="image" uuid="840c6f79-54b7-4b33-a18f-64cffbd19799"/>
Apr 14 04:53:15 user nova-compute[71317]:       <nova:ports>
Apr 14 04:53:15 user nova-compute[71317]:         <nova:port uuid="2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b">
Apr 14 04:53:15 user nova-compute[71317]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 14 04:53:15 user nova-compute[71317]:         </nova:port>
Apr 14 04:53:15 user nova-compute[71317]:       </nova:ports>
Apr 14 04:53:15 user nova-compute[71317]:     </nova:instance>
Apr 14 04:53:15 user nova-compute[71317]:   </metadata>
Apr 14 04:53:15 user nova-compute[71317]:   <sysinfo type="smbios">
Apr 14 04:53:15 user nova-compute[71317]:     <system>
Apr 14 04:53:15 user nova-compute[71317]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 04:53:15 user nova-compute[71317]:       <entry name="product">OpenStack Nova</entry>
Apr 14 04:53:15 user nova-compute[71317]:       <entry name="version">0.0.0</entry>
Apr 14 04:53:15 user nova-compute[71317]:       <entry name="serial">068de634-b290-46ae-b5a2-ef36165cd23b</entry>
Apr 14 04:53:15 user nova-compute[71317]:       <entry name="uuid">068de634-b290-46ae-b5a2-ef36165cd23b</entry>
Apr 14 04:53:15 user nova-compute[71317]:       <entry name="family">Virtual Machine</entry>
Apr 14 04:53:15 user nova-compute[71317]:     </system>
Apr 14 04:53:15 user nova-compute[71317]:   </sysinfo>
Apr 14 04:53:15 user nova-compute[71317]:   <os>
Apr 14 04:53:15 user nova-compute[71317]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 04:53:15 user nova-compute[71317]:     <boot dev="hd"/>
Apr 14 04:53:15 user nova-compute[71317]:     <smbios mode="sysinfo"/>
Apr 14 04:53:15 user nova-compute[71317]:   </os>
Apr 14 04:53:15 user nova-compute[71317]:   <features>
Apr 14 04:53:15 user nova-compute[71317]:     <acpi/>
Apr 14 04:53:15 user nova-compute[71317]:     <apic/>
Apr 14 04:53:15 user nova-compute[71317]:     <vmcoreinfo/>
Apr 14 04:53:15 user nova-compute[71317]:   </features>
Apr 14 04:53:15 user nova-compute[71317]:   <clock offset="utc">
Apr 14 04:53:15 user nova-compute[71317]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 04:53:15 user nova-compute[71317]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 04:53:15 user nova-compute[71317]:     <timer name="hpet" present="no"/>
Apr 14 04:53:15 user nova-compute[71317]:   </clock>
Apr 14 04:53:15 user nova-compute[71317]:   <cpu mode="custom" match="exact">
Apr 14 04:53:15 user nova-compute[71317]:     <model>Nehalem</model>
Apr 14 04:53:15 user nova-compute[71317]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 04:53:15 user nova-compute[71317]:   </cpu>
Apr 14 04:53:15 user nova-compute[71317]:   <devices>
Apr 14 04:53:15 user nova-compute[71317]:     <disk type="file" device="disk">
Apr 14 04:53:15 user nova-compute[71317]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 04:53:15 user nova-compute[71317]:       <source file="/opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk"/>
Apr 14 04:53:15 user nova-compute[71317]:       <target dev="vda" bus="virtio"/>
Apr 14 04:53:15 user nova-compute[71317]:     </disk>
Apr 14 04:53:15 user nova-compute[71317]:     <interface type="ethernet">
Apr 14 04:53:15 user nova-compute[71317]:       <mac address="fa:16:3e:65:0b:60"/>
Apr 14 04:53:15 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:53:15 user nova-compute[71317]:       <mtu size="1442"/>
Apr 14 04:53:15 user nova-compute[71317]:       <target dev="tap2cd60678-2b"/>
Apr 14 04:53:15 user nova-compute[71317]:     </interface>
Apr 14 04:53:15 user nova-compute[71317]:     <serial type="pty">
Apr 14 04:53:15 user nova-compute[71317]:       <log file="/opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/console.log" append="off"/>
Apr 14 04:53:15 user nova-compute[71317]:     </serial>
Apr 14 04:53:15 user nova-compute[71317]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 04:53:15 user nova-compute[71317]:     <video>
Apr 14 04:53:15 user nova-compute[71317]:       <model type="virtio"/>
Apr 14 04:53:15 user nova-compute[71317]:     </video>
Apr 14 04:53:15 user nova-compute[71317]:     <rng model="virtio">
Apr 14 04:53:15 user nova-compute[71317]:       <backend model="random">/dev/urandom</backend>
Apr 14 04:53:15 user nova-compute[71317]:     </rng>
Apr 14 04:53:15 user nova-compute[71317]:     <controller type="usb" index="0" model="none"/>
Apr 14 04:53:15 user nova-compute[71317]:     <memballoon model="virtio">
Apr 14 04:53:15 user nova-compute[71317]:       <stats period="10"/>
Apr 14 04:53:15 user nova-compute[71317]:     </memballoon>
Apr 14 04:53:15 user nova-compute[71317]:   </devices>
Apr 14 04:53:15 user nova-compute[71317]: </domain>
Apr 14 04:53:15 user nova-compute[71317]:  {{(pid=71317) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:49:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-203487374',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-203487374',id=47,image_ref='840c6f79-54b7-4b33-a18f-64cffbd19799',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-keypair-1115532707',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:49:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='c10c8cf348e04e44afbaae93a4b60e36',ramdisk_id='',reservation_id='r-p7q5eq40',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2141578557',owner_user_name='tempest-AttachVolumeShelveTestJSON-2141578557-project-member',shelved_at='2023-04-14T04:51:20.505352',shelved_host='user',shelved_image_id='840c6f79-54b7-4b33-a18f-64cffbd19799'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T04:53:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='67d6c699fc524554a6f4983af91d58fc',uuid=068de634-b290-46ae-b5a2-ef36165cd23b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converting VIF {"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG os_vif [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b') {{(pid=71317) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2cd60678-2b, may_exist=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2cd60678-2b, col_values=(('external_ids', {'iface-id': '2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:65:0b:60', 'vm-uuid': '068de634-b290-46ae-b5a2-ef36165cd23b'}),)) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:15 user nova-compute[71317]: INFO os_vif [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b')
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] No BDM found with device name vda, not building metadata. {{(pid=71317) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 04:53:15 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] No VIF found with MAC fa:16:3e:65:0b:60, not building metadata {{(pid=71317) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 04:53:16 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:16 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Cleaning up deleted instances with incomplete migration  {{(pid=71317) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 04:53:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:16 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-68ab0931-6d3f-4b97-a71f-f3eaeac68a0a req-31052a73-716e-4afc-9753-41b977cae982 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:53:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-68ab0931-6d3f-4b97-a71f-f3eaeac68a0a req-31052a73-716e-4afc-9753-41b977cae982 service nova] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:53:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-68ab0931-6d3f-4b97-a71f-f3eaeac68a0a req-31052a73-716e-4afc-9753-41b977cae982 service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:53:17 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-68ab0931-6d3f-4b97-a71f-f3eaeac68a0a req-31052a73-716e-4afc-9753-41b977cae982 service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:53:17 user nova-compute[71317]: DEBUG nova.compute.manager [req-68ab0931-6d3f-4b97-a71f-f3eaeac68a0a req-31052a73-716e-4afc-9753-41b977cae982 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] No waiting events found dispatching network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:53:17 user nova-compute[71317]: WARNING nova.compute.manager [req-68ab0931-6d3f-4b97-a71f-f3eaeac68a0a req-31052a73-716e-4afc-9753-41b977cae982 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received unexpected event network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b for instance with vm_state shelved_offloaded and task_state spawning.
Apr 14 04:53:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:17 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:18 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447998.8731139, 068de634-b290-46ae-b5a2-ef36165cd23b => Resumed> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:53:18 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] VM Resumed (Lifecycle Event)
Apr 14 04:53:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Instance event wait completed in 0 seconds for  {{(pid=71317) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 04:53:18 user nova-compute[71317]: DEBUG nova.virt.libvirt.driver [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Guest created on hypervisor {{(pid=71317) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 04:53:18 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Instance spawned successfully.
Apr 14 04:53:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:53:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:53:18 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:53:18 user nova-compute[71317]: DEBUG nova.virt.driver [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] Emitting event <LifecycleEvent: 1681447998.8745868, 068de634-b290-46ae-b5a2-ef36165cd23b => Started> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:53:18 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] VM Started (Lifecycle Event)
Apr 14 04:53:18 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:53:18 user nova-compute[71317]: DEBUG nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=71317) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 04:53:18 user nova-compute[71317]: INFO nova.compute.manager [None req-d5bb74e4-0e5d-49ca-8265-dc4899226278 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 04:53:19 user nova-compute[71317]: DEBUG nova.compute.manager [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:53:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-f0918c56-ec8a-4afe-8be7-b9c1131fe897 req-cc64562d-5176-4bbb-afc8-3eac47a8f49b service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:53:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0918c56-ec8a-4afe-8be7-b9c1131fe897 req-cc64562d-5176-4bbb-afc8-3eac47a8f49b service nova] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:53:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0918c56-ec8a-4afe-8be7-b9c1131fe897 req-cc64562d-5176-4bbb-afc8-3eac47a8f49b service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:53:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-f0918c56-ec8a-4afe-8be7-b9c1131fe897 req-cc64562d-5176-4bbb-afc8-3eac47a8f49b service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:53:19 user nova-compute[71317]: DEBUG nova.compute.manager [req-f0918c56-ec8a-4afe-8be7-b9c1131fe897 req-cc64562d-5176-4bbb-afc8-3eac47a8f49b service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] No waiting events found dispatching network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:53:19 user nova-compute[71317]: WARNING nova.compute.manager [req-f0918c56-ec8a-4afe-8be7-b9c1131fe897 req-cc64562d-5176-4bbb-afc8-3eac47a8f49b service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received unexpected event network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b for instance with vm_state shelved_offloaded and task_state spawning.
Apr 14 04:53:19 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-c4b7373a-6318-4a02-b4ed-29a96c3fcf0e tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "068de634-b290-46ae-b5a2-ef36165cd23b" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 9.672s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:53:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:23 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:53:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:34 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_power_states {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:34 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Triggering sync for uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 04:53:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:53:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "068de634-b290-46ae-b5a2-ef36165cd23b" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:53:34 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "068de634-b290-46ae-b5a2-ef36165cd23b" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.025s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:53:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:38 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:53:48 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:50 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:53:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:53:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:53:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:53:50 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:53:50 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lazy-loading 'info_cache' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:53:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:50 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating instance_info_cache with network_info: [{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:53:50 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:53:50 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:53:50 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:54 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:55 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:53:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:53:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:53:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:53:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:53:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:53:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:53:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:53:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:53:59 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:53:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:53:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:53:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 04:53:59 user nova-compute[71317]: DEBUG oslo_concurrency.processutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71317) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 04:53:59 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:53:59 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:53:59 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8810MB free_disk=26.223690032958984GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:53:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:53:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:54:00 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 068de634-b290-46ae-b5a2-ef36165cd23b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:54:00 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:54:00 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:54:00 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:54:00 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:54:00 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:54:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.219s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:54:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:54:02 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:54:02 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:54:02 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:54:02 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:54:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:54:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:54:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:54:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:54:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:54:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:54:33 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:54:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:54:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:54:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:54:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:50 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:54:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:54:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:54:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:54:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:54:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:52 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:54:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:54:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:54:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 04:54:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquired lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 04:54:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Forcefully refreshing network info cache for instance {{(pid=71317) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 04:54:52 user nova-compute[71317]: DEBUG nova.objects.instance [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lazy-loading 'info_cache' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:54:52 user nova-compute[71317]: DEBUG nova.network.neutron [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating instance_info_cache with network_info: [{"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:54:52 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Releasing lock "refresh_cache-068de634-b290-46ae-b5a2-ef36165cd23b" {{(pid=71317) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 04:54:52 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updated the network info_cache for instance {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 04:54:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:56 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:54:56 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "068de634-b290-46ae-b5a2-ef36165cd23b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:54:59 user nova-compute[71317]: INFO nova.compute.manager [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Terminating instance
Apr 14 04:54:59 user nova-compute[71317]: DEBUG nova.compute.manager [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Start destroying the instance on the hypervisor. {{(pid=71317) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:59 user nova-compute[71317]: INFO nova.virt.libvirt.driver [-] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Instance destroyed successfully.
Apr 14 04:54:59 user nova-compute[71317]: DEBUG nova.objects.instance [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lazy-loading 'resources' on Instance uuid 068de634-b290-46ae-b5a2-ef36165cd23b {{(pid=71317) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG nova.virt.libvirt.vif [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T04:49:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-203487374',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-203487374',id=47,image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIz9kPiXrU+d9YNNeLmDz42lc/lqPHsJkMzNHXLJ9BxaT3PcpDfhDBMRUsgkcRE+o41EINj1zxZFQZUWhpX4XIsgWigDmt6Wb2qr1dlIExZrdUiB9GwXx8T2hNCZNX55JA==',key_name='tempest-keypair-1115532707',keypairs=<?>,launch_index=0,launched_at=2023-04-14T04:53:19Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c10c8cf348e04e44afbaae93a4b60e36',ramdisk_id='',reservation_id='r-p7q5eq40',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader',clean_attempts='1',image_base_image_ref='5c55a1da-f6f8-4442-939b-f0cb72ea4327',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2141578557',owner_user_name='tempest-AttachVolumeShelveTestJSON-2141578557-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T04:53:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='67d6c699fc524554a6f4983af91d58fc',uuid=068de634-b290-46ae-b5a2-ef36165cd23b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converting VIF {"id": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "address": "fa:16:3e:65:0b:60", "network": {"id": "4c4f373b-6540-48ea-815b-d4930f403496", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-2101199077-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c10c8cf348e04e44afbaae93a4b60e36", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cd60678-2b", "ovs_interfaceid": "2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG nova.network.os_vif_util [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b') {{(pid=71317) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG os_vif [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b') {{(pid=71317) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:54:59 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2cd60678-2b, bridge=br-int, if_exists=True) {{(pid=71317) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:55:00 user nova-compute[71317]: INFO os_vif [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:65:0b:60,bridge_name='br-int',has_traffic_filtering=True,id=2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b,network=Network(4c4f373b-6540-48ea-815b-d4930f403496),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cd60678-2b')
Apr 14 04:55:00 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Deleting instance files /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b_del
Apr 14 04:55:00 user nova-compute[71317]: INFO nova.virt.libvirt.driver [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Deletion of /opt/stack/data/nova/instances/068de634-b290-46ae-b5a2-ef36165cd23b_del complete
Apr 14 04:55:00 user nova-compute[71317]: INFO nova.compute.manager [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Took 0.66 seconds to destroy the instance on the hypervisor.
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo.service.loopingcall [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71317) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.compute.manager [-] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Deallocating network for instance {{(pid=71317) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] deallocate_for_instance() {{(pid=71317) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-6bf28e45-2706-41a1-85e6-98a9bba52ff6 req-dfebed9e-701b-4a46-b749-054a041411be service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-vif-unplugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6bf28e45-2706-41a1-85e6-98a9bba52ff6 req-dfebed9e-701b-4a46-b749-054a041411be service nova] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6bf28e45-2706-41a1-85e6-98a9bba52ff6 req-dfebed9e-701b-4a46-b749-054a041411be service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-6bf28e45-2706-41a1-85e6-98a9bba52ff6 req-dfebed9e-701b-4a46-b749-054a041411be service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-6bf28e45-2706-41a1-85e6-98a9bba52ff6 req-dfebed9e-701b-4a46-b749-054a041411be service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] No waiting events found dispatching network-vif-unplugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-6bf28e45-2706-41a1-85e6-98a9bba52ff6 req-dfebed9e-701b-4a46-b749-054a041411be service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-vif-unplugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b for instance with task_state deleting. {{(pid=71317) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:00 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:55:00 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8849MB free_disk=26.242919921875GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Instance 068de634-b290-46ae-b5a2-ef36165cd23b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71317) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.198s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.network.neutron [-] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Updating instance_info_cache with network_info: [] {{(pid=71317) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 04:55:00 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Took 0.79 seconds to deallocate network for instance.
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:55:00 user nova-compute[71317]: DEBUG nova.compute.manager [req-76a8583f-978d-45bc-85b1-2a7ea0916e7e req-907a1000-cd4f-4cca-b085-317a831e571e service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-vif-deleted-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:55:01 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:55:01 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:55:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.187s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:55:01 user nova-compute[71317]: INFO nova.scheduler.client.report [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Deleted allocations for instance 068de634-b290-46ae-b5a2-ef36165cd23b
Apr 14 04:55:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-624dcbc6-fd7c-47ef-b2cb-6d1eecf8cb67 tempest-AttachVolumeShelveTestJSON-2141578557 tempest-AttachVolumeShelveTestJSON-2141578557-project-member] Lock "068de634-b290-46ae-b5a2-ef36165cd23b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.819s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:55:01 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:55:01 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:55:01 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:55:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-7782bc07-ac99-403c-8258-3512beda3c2b req-25beeb66-f322-4f4a-873d-33a69578c857 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received event network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 04:55:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7782bc07-ac99-403c-8258-3512beda3c2b req-25beeb66-f322-4f4a-873d-33a69578c857 service nova] Acquiring lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:55:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7782bc07-ac99-403c-8258-3512beda3c2b req-25beeb66-f322-4f4a-873d-33a69578c857 service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:55:02 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [req-7782bc07-ac99-403c-8258-3512beda3c2b req-25beeb66-f322-4f4a-873d-33a69578c857 service nova] Lock "068de634-b290-46ae-b5a2-ef36165cd23b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:55:02 user nova-compute[71317]: DEBUG nova.compute.manager [req-7782bc07-ac99-403c-8258-3512beda3c2b req-25beeb66-f322-4f4a-873d-33a69578c857 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] No waiting events found dispatching network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b {{(pid=71317) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 04:55:02 user nova-compute[71317]: WARNING nova.compute.manager [req-7782bc07-ac99-403c-8258-3512beda3c2b req-25beeb66-f322-4f4a-873d-33a69578c857 service nova] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Received unexpected event network-vif-plugged-2cd60678-2bf1-45e0-8ca7-cf8327cc9a5b for instance with vm_state deleted and task_state None.
Apr 14 04:55:03 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:55:03 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:55:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:55:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:55:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:55:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:14 user nova-compute[71317]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681448099.9712584, 068de634-b290-46ae-b5a2-ef36165cd23b => Stopped> {{(pid=71317) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 04:55:14 user nova-compute[71317]: INFO nova.compute.manager [-] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] VM Stopped (Lifecycle Event)
Apr 14 04:55:14 user nova-compute[71317]: DEBUG nova.compute.manager [None req-495b81d3-b484-4601-9c35-d6d97fbc5074 None None] [instance: 068de634-b290-46ae-b5a2-ef36165cd23b] Checking state {{(pid=71317) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 04:55:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:17 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:55:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:55:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:55:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:55:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71317) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 04:55:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:55:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71317) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 04:55:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:52 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:55:54 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:55:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:55:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:55:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Didn't find any instances for network info cache update. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 04:55:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:55:58 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:55:58 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:56:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:56:00 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:56:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:56:01 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:56:01 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:56:01 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8881MB free_disk=26.241737365722656GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:56:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.154s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:56:03 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:56:03 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:56:03 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:56:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:56:05 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:56:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:56:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:56:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:56:25 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:56:30 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:56:35 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:56:40 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:56:45 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:56:50 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:56:53 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:56:54 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:56:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Starting heal instance info cache {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 04:56:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Rebuilding the list of instances to heal {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 04:56:54 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Didn't find any instances for network info cache update. {{(pid=71317) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 04:56:55 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 04:56:59 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:57:00 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:57:00 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager.update_available_resource {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Auditing locally available compute resources for user (node: user) {{(pid=71317) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 04:57:01 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:57:01 user nova-compute[71317]: WARNING nova.virt.libvirt.driver [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 04:57:01 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Hypervisor/Node resource view: name=user free_ram=8865MB free_disk=26.230972290039062GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}] {{(pid=71317) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71317) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG nova.compute.provider_tree [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed in ProviderTree for provider: e35a99e4-b1ea-4792-a9ff-f93045b2018b {{(pid=71317) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG nova.scheduler.client.report [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Inventory has not changed for provider e35a99e4-b1ea-4792-a9ff-f93045b2018b based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71317) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG nova.compute.resource_tracker [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Compute_service record updated for user:user {{(pid=71317) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 04:57:01 user nova-compute[71317]: DEBUG oslo_concurrency.lockutils [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.121s {{(pid=71317) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 04:57:02 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:57:04 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:57:04 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:57:04 user nova-compute[71317]: DEBUG nova.compute.manager [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71317) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 04:57:04 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:57:05 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:57:05 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 04:57:10 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:57:15 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:57:20 user nova-compute[71317]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71317) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 04:57:21 user nova-compute[71317]: DEBUG oslo_service.periodic_task [None req-a49102c1-42f1-420e-814f-92375b4ce99c None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71317) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}